Positional tracking of harvester within field

ABSTRACT

A harvester travels along a route within a field and selectively harvests edible crowns ready for harvesting. As the harvester travels along the route, a position of the harvester and/or the edible crowns may be determined using an encoder, imaging device(s), and/or navigational system(s). For example, image(s) captured by the imaging device(s) may be used to determine a location of the edible crowns and/or global positioning satellite (GPS) coordinates may indicate a location of the harvester within the field. These locations may be used for instructing harvesting components to harvest the edible crowns. For example, the harvester may include robotic arms having end effectors that harvest the edible crowns. Knowing the location of the edible crowns and/or the harvester therefore allows for the accurate placement of the end effectors for harvesting the edible crowns.

BACKGROUND

Several vegetable Crops are normally harvested by hand. Cauliflower,broccoli, or other stemmed vegetables, for example, are usuallyharvested manually by a crew of workers. As part of this process,workers visually inspect each plant to determine whether the plant isready for harvesting. Conventional techniques also involve multiplehandling stages, which gives rise to bruising or damage. The process ofexamining, harvesting, and sorting individual plants is labor-intensive,inefficient, and wasteful.

Attempts have been made to automate or semi-automate harvesting.However, given the varying nature of the plants and/or field conditionsat a time of harvesting (e.g., size, shape, mud, dust etc.), challengesstill remain to observe a harvestable portion of the plant, gripping theplants, and/or separating edible portions from non-edible portionssufficient for automated or semi-automated harvest. Automated harvestingalso may require additional manual processing, thus negating somebenefits. Further technological improvements may increase harvest yieldsand efficiencies while reducing waste and manual labor.

BRIEF DESCRIPTION OF THE DRAWINGS

The detailed description is set forth with reference to the accompanyingfigures. In the figures, the left-most digit(s) of a reference numberidentifies the figure in which the reference number first appears. Theuse of the same reference numbers in different figures indicates similaror identical items or features. The systems and devices depicted in theaccompanying figures are not to scale and components within the figuresmay be depicted not to scale with each other.

FIG. 1 illustrates a first perspective view of an example harvester forharvesting crops, such as broccoli, according to an embodiment of thepresent disclosure. In some instances, the harvester may include imagingcomponents for imaging parts of broccoli plants for use in determiningwhether an edible crown of the broccoli plant is ready for harvesting(e.g., ripe). If ready, harvesting components may harvest (e.g., pick)the edible crown.

FIG. 2 illustrates a second perspective view of the harvester of FIG. 1,according to an embodiment of the present disclosure.

FIG. 3 illustrates a first side view of the harvester of FIG. 1,according to embodiment of the present disclosure.

FIG. 4 illustrates a second side view of the harvester of FIG. 1,according to an embodiment of the present disclosure.

FIG. 5 illustrates example components of the harvester of FIG. 1,according to an embodiment of the present disclosure.

FIG. 6A illustrates a diagram showing an example route of the harvesterof FIG. 1 within a field for harvesting edible crowns, according to anembodiment of the present disclosure.

FIG. 6B illustrates the harvester moving along the route of FIG. 6A forharvesting edible crowns, according to an embodiment of the presentdisclosure. The harvester may image the edible crowns, determine whetherthe edible crowns are ready for harvesting, and if so, may harvest theedible crowns.

FIG. 7 illustrates an example imaging system of the harvester of FIG. 1for use in determining whether the edible crowns are ready forharvesting, according to an embodiment of the present disclosure.

FIG. 8A illustrates a diagram for aligning example harvesting componentsof the harvester of FIG. 1 to harvest edible crowns, according to anembodiment of the present disclosure. The diagram of FIG. 8A illustratesaligning the harvesting components with the edible crown in a firstdirection and a second direction, or along a first axis/plane and asecond axis/plane, respectively.

FIG. 8B illustrates a diagram for aligning example harvesting componentsof the harvester of FIG. 1 to harvest edible crowns, according to anembodiment of the present disclosure. The diagram of FIG. 8B illustratesaligning the harvesting components in a third direction with the ediblecrown, or along a third axis/plane.

FIG. 9 illustrates an example diagram for aligning harvesting componentsof the harvester of FIG. 1 to harvest edible crowns, according to anembodiment of the present disclosure.

FIG. 10 illustrates a diagram showing an example route of the harvesterof FIG. 1 for harvesting edible crowns, according to an embodiment ofthe present disclosure.

FIG. 11A illustrates an example de-leafing component of the harvester ofFIG. 1 for removing leaves on the broccoli plant, according to anembodiment of the present disclosure.

FIG. 11B illustrates the de-leafing component of FIG. 11A, according toan embodiment of the present disclosure.

FIG. 12A illustrates a first view of example harvesting components ofthe harvester of FIG. 1, according to an embodiment of the presentdisclosure. In some instances, the harvesting components may include arobotic arm and/or an end effector for grasping the edible crown and acutting mechanism for severing the edible crown from the broccoli plant.The end effector may operably transition between an open state and aclosed state for grasping the edible crown and releasing the ediblecrown at a collection point. In FIG. 12A, the end effector is in theclosed state.

FIG. 12B illustrates a second view of the harvesting components of FIG.12A, according to an embodiment of the present disclosure. In FIG. 12B,the end effector is in the open state.

FIG. 13A illustrates an example transferring component of the harvesterof FIG. 1 for receiving harvested edible crowns from harvestingcomponents, and transferring the edible crown(s) to other portions ofthe harvester, such as a conveyor belt, according to an embodiment ofthe present disclosure. The transferring component may represent aflipper that operably transitions between different positions, such as afirst position for receiving harvested edible crowns from the endeffector, and a second position, for transferring the edible crowns. InFIG. 13A, the flipper is in the first position for receiving theharvested edible crowns.

FIG. 13B illustrates the transferring component of FIG. 13A, accordingto an embodiment of the present disclosure. In FIG. 13B, the flipper isin the second position for transferring the harvested edible crowns.

FIG. 14 illustrates an example process for training one or moremachine-learning model(s) for use in determining whether an edible crownis ready for harvesting, according to an embodiment of the presentdisclosure.

FIG. 15 illustrates an example process for determining whether an ediblecrown is ready for harvesting, according to an embodiment of the presentdisclosure.

FIG. 16 illustrates an example process for determining whether toharvest an edible crown, according to an embodiment of the presentdisclosure.

FIG. 17 illustrates an example process for harvesting edible crowns,including aligning harvesting components with an edible crown ready forharvesting, according to an embodiment of the present disclosure.

FIG. 18 illustrates an example process for determining a position orcoordinates of an edible crown ready for harvesting, according to anembodiment of the present disclosure.

FIG. 19 illustrates an example process for adjusting a speed of aharvester based on a number of harvestable edible crowns across rows ofbroccoli plants within a field, according to an embodiment of thepresent disclosure.

FIG. 20 illustrates an example process for determining a type of cut foredible crowns ready for harvesting, according to an embodiment of thepresent disclosure.

FIG. 21A illustrates a side view of example harvesting components of theharvester of FIG. 1, according to an embodiment of the presentdisclosure. In some instances, the harvesting components may include arobotic arm and/or an end effector for grasping the edible crown and acutting mechanism for severing the edible crown from the broccoli plant.The end effector may operably transition between an open state and aclosed state for grasping the edible crown and releasing the ediblecrown at a collection point. In FIG. 21A, the end effector is in theclosed state.

FIG. 21B illustrates a side view of the harvesting components of FIG.21A, according to an embodiment of the present disclosure. In FIG. 21B,the end effector is in the open state.

FIG. 21C illustrates a perspective view of the harvesting components ofFIG. 21A, according to an embodiment of the present disclosure. In FIG.21C, the end effector is in the closed state.

FIG. 21D illustrates a perspective view of the harvesting components ofFIG. 21A, according to an embodiment of the present disclosure. In FIG.21D, the end effector is in the open state.

FIG. 21E illustrates an end view of the harvesting components of FIG.21A, according to an embodiment of the present disclosure. In FIG. 21E,the end effector is in the closed state.

DETAILED DESCRIPTION

Described herein are, among other things, techniques, devices, andsystems for determining characteristics of edible crowns using machinelearning and determining whether the edible crowns are ready forharvesting. In some instances, a machine or harvester selectively androbotically harvests the edible crowns that are ready for harvesting.The harvester may include sensors, such as an imaging system, fordetecting and analyzing characteristic(s) of edible crowns of thebroccoli plants (e.g., head, flower, floret, crown, edible portion etc.)for use in controlling mechanical pickers that harvest the ediblecrowns. For example, the harvester may include robotic arms that mayhave end effectors that function to mechanically separate the ediblecrowns from a remainder of the broccoli plant (e.g., the stem, stalk,leaves, etc.). In some instances, the harvester may include any numberof robotic arms for harvesting the edible crowns across multiple rows ofbroccoli plants. As the harvester maneuvers through a field of broccoliplants arranged in rows, the harvester may detect which of the ediblecrowns are ready for harvesting, remove (e.g., trim, strip, etc.) leavesfrom the stem and/or around/on the edible crown, separate the ediblecrown from the rest of the stem, and collect the edible crowns at one ormore collection points. The techniques and systems described herein mayprovide improved efficiencies for harvesting edible crowns, reducingwaste, and increasing yields.

In some instances, the harvester may represent a self-propelledautomated platform or platform that is towed, pulled, pushed, or carriedby a tractor, for example. The platform may provide a space or areaoccupied by one or more operators, workers, and/or one or more foreman.The harvester may include a body, or frame, having wheels or trackswhich engage with the ground for traversing over landscapes or terrain(e.g., crops, fields, etc.). The frame may reside vertically above thebroccoli plants such that the broccoli plants pass underneath the frame,between the wheels of the harvester, as the harvester moves about thefield. In instances where the harvester is self-propelled, the harvestermay include a driving mechanism (e.g., engines, motors, transmissions,gears, generators, etc.) that power the wheels for moving across thefield.

A navigational system of the harvester may be utilized to navigate theharvester throughout the field. In some instances, the navigationalsystem may include a global positioning satellite (GPS) system or otherlocation-based tracking system for navigating the harvester throughoutthe field. In some instances, the navigational system may be used forcontrolling the harvester and guiding the harvester along apredetermined route or path for harvesting the edible crowns. Forexample, the predetermined route may include a serpentine path thattraverses the rows of broccoli plants within the field. In someinstances, the navigational system may communicatively couple to asteering device and/or the driving mechanism for providing instructionsand maneuvering the harvester along the predetermined route or path.

In some instances, the harvester may include a de-leafing component thatremoves leaves of the broccoli plant. For example, broccoli plantstypically have an abundance of leaves that grow from the broccoli stemand which reside beneath, alongside of, and even above the edible crown.These leaves may conceal the edible crowns and impact a quality ofimage(s) obtained of the edible crown. If the leaves were notsubstantially removed before imaging the crowns, the computing systemmay inaccurately determine, or be unable to determine, whether theedible crown is ready for harvesting. Accordingly, in some instances,the de-leafing component may be positioned in front of the imagingsystem, relative to the direction of travel of the harvester, to removethe leaves and isolate or expose the edible crown. Therein, the imagingsystem may image the edible crown for use by the computing system todetermine whether the edible crown is ready for harvesting.

The harvester may be configured to harvest the edible crowns dependingon the maturity of individual broccoli plants (or the individual ediblecrowns). For example, the harvester may selectively harvest the ediblecrowns based on whether the edible crowns are ripe for picking (e.g.,mature, full-grown, etc.). To assist in this process, the harvesterincludes components for determining whether the edible crowns are readyto be harvested, and in such cases, includes components for harvestingthe edible crowns and/or processing the edible crowns (e.g., packaging,cleaning, trimming, etc.). The components may be distributed or mountedacross the platform and/or the frame of the harvester. In someinstances, the entirety of the process for harvesting the edible crownsmay be automated and/or workers may assist in harvesting the ediblecrowns. For example, the workers may clean and/or package the ediblecrowns once harvested.

In some instances, the harvester may include an imaging system fordetecting harvestable edible crowns within the field. For example, theimaging system may image unharvested rows of broccoli plants within thefield as the harvester maneuvers within the field. The imaging systemmay be positioned vertically above the broccoli plants and arranged toimage the edible crowns from above. As the broccoli plants (or theedible crowns) come within a field of view of the imaging system,image(s) of the edible crowns may be captured. In some instances, theimaging system may continuously image the broccoli plants such that aseries of images of the edible crowns are obtained. However, in someinstances, a single image of individual edible crowns may be obtained.

As the harvester moves across the field the edible crowns of the rows ofbroccoli plants may come into view of the imaging system and the imagingsystem may image a succeeding or next edible crown for determiningwhether the next edible crown is ready for harvesting. In someinstances, the imaging system may be at a fixed position on theharvester or the imaging system may be actuatable to position and/or aimat the edible crowns. In some instances, the imaging system may beangled in such a way to capture ensuing or successive edible crowns in adirection of travel of the harvester. In this manner, the harvester maycontinuously image the edible crowns as the harvester moves about thefield.

The harvester may also include lighting element(s) that illuminate thebroccoli plants, such as the edible crowns, for obtaining clear imagesand/or increased image quality. Illuminating the edible crowns mayincrease the identification of harvestable edible crowns, such asobtaining images depicting a color of the edible crowns and/or a size ofthe edible crowns (e.g., distinguishing the edible crown from otherportions of the broccoli plant and/or the environment). The lightingelement(s) may also permit the harvester to be operated in low-light, orno natural light conditions, such as dusk, dawn, and/or at night.

In some instances, the imaging system may include one or more camerasand/or one or more depth sensors for imaging the edible crowns. In someinstances, the one or more cameras may include red-green-blue (RGB)cameras and the one or more depth sensors may include infrared (IR)sensors. As the harvester may be configured to harvest more than one rowof broccoli plants simultaneously, or edible crowns across multiple rowsof broccoli plants, the harvester may include imaging systems for therespective rows. By way of example, the harvester may be configured tosimultaneously harvest twelve (or more) rows of broccoli plants, and insuch instances, the harvester may include twelve (or more) imagingsystems for detecting the harvestable edible crowns within theindividual rows of broccoli plants. A first imaging system may imageedible crowns within a first row, a second imaging system may imageedible crowns within a second row, and so forth. However, in someinstances, the imaging systems may image more than one row of broccoliplants, or across rows of broccoli plants. For example, an imagingsystem may include a first camera for imaging a first row of broccoliplants and a second camera for imaging a second row of broccoli plants.Additionally, or alternatively, a single camera of the imaging systemmay image a first row of the broccoli plants and a second row of thebroccoli plants.

As discussed above, the harvester may be configured to harvest theedible crowns based on the characteristic(s) or properties of thebroccoli plant (or the edible crown). For example, based on the imagingperformed by the imaging system, a computing system of the harvester mayanalyze the image(s) for determining characteristic(s) of the individualbroccoli plants, or parts thereof (e.g., the edible crown, stem, leaves,etc.). By way of example, using the image(s) captured by the imagingsystem, the computing system may analyze the image(s) to determine asize, color, condition, quality, health, and/or ripeness of the ediblecrown. The computing system may be configured to process, in parallel,image(s) captured by the multiple imaging system for determiningcharacteristic(s) of the broccoli plants and whether to respectivelyharvest edible crowns within the rows of broccoli plants. In someinstances, these characteristics may be compared against referencecharacteristics to determine whether the edible crowns are ready forharvesting.

In some instances, the computing system may determine, based on thecharacteristic(s), a probability (or score) that the edible crown isready for harvesting. If the probability satisfies a certain confidencethreshold, the edible crown may be deemed or determined ready forharvesting. Additionally, or alternatively, in some instances, todetermine the probability, the computing system may utilizemachine-learning (ML) model(s). For example, the ML model(s) may betrained from a database (e.g., historical data, such as image data, ofpast edible crowns that were harvested, or past edible crowns that werenot harvested) to analyze the image(s) captured by the imaging systemfor identifying characteristic(s) of the edible crown, such as color,shape, size, etc. The ML model(s), upon identifying one or more of thesecharacteristic(s), may assess the color, shape, size, etc. in comparisonwith information stored in the database to determine whether the ediblecrown is ready for harvesting.

Noted above, the database may be previously trained (e.g., via the MLmodel(s)) to indicate characteristic(s) of the edible crowns that areassociated with edible crowns ready for harvesting. For example, thetrained database may indicate a range of colors associated with ediblecrowns that are ready for harvesting, and upon the ML model(s)determining the color of the edible crown (e.g., average color), the MLmodel(s) may compare this color to the colors in the database fordetermining whether the edible crown is ready for harvesting. In someinstances, these colors in the database may be considered referencecolors to which the identified color by the ML model(s) is compared.However, in some instances, the computing system may additionally oralternatively use a trained database indicative of characteristic(s) ofedible crowns that are not ready for harvesting to determine to notharvest edible crown(s).

As part of the ML model(s) analyzing the image(s), the ML model(s) maylabel characteristic(s) of the edible crown that indicate whether thecharacteristic is associated with an edible crown that is ready forharvesting. Therein, an output of the ML model(s) may indicate whetherthe edible crown is ready for harvesting. In some instances, the MLmodel(s) may determine the characteristics(s) for comparison torespective references for determining whether the edible crown is readyfor harvesting. As such, the harvester (or a remotely coupled computingdevice) may store a database of information indicative of properties ofthe edible crown that are ready for harvesting.

However, the ML model(s) may use any number of characteristic(s) fordetermining whether to harvest the edible crown. For example, the MLmodel(s) may use or determine any number of characteristic(s) orproperties of the imaged edible crowns for use in determining whetherthe edible crowns are ready for harvesting or not ready for harvesting(e.g., one, two, three, four, etc.). In such instances, the ML model(s)may weigh certain characteristics relative to others. For example, acolor of the edible crown may be more indicative of the edible crownbeing ready for harvesting, as compared to a size of the edible crown.As such, after determining characteristic(s) of the imaged edible crown,the characteristic(s) may be compared against reference characteristicsto determine whether the edible crown is ready for harvesting. The MLmodel(s) may output the probability that represents whether the ediblecrown is ready for harvesting, based on the analyzed characteristic(s).

Utilizing the imaging system, the computing system may therefore assessthe maturity, or immaturity, of the edible crowns such that thecomputing system may differentiate between mature and immature ediblecrowns. Edible crowns that are mature may be considered harvestable,while edible crowns that are immature may not be deemed harvestable (ornot mature for harvesting). For example, because broccoli plants matureat different rates, on a given day, some edible crowns may be ready toharvest while other edible crowns may not be ready to harvest. Theedible crowns that are not ready for harvesting are left in the groundfor harvesting at a later time. Processing the image(s) captured by theimaging system and using the ML model(s) may therefore be used to selectwhich edible crowns are to be harvested. Knowing which edible crowns areharvestable permits the harvester to selectively harvest these ediblecrowns. As such, the computing system may cause these edible crowns tobe harvested.

For example, and in some instances, the harvester may include automatedmechanical components or robotic arms having end effectors, such as agripper, for harvesting the edible crowns. The robotic arms may belocated behind the imaging systems, in a direction of travel of theharvester, such that results of the imaging system may be used todetermine whether to harvest the edible crowns. In these sense, theimaging systems may be located in front of robotic arms such that theharvester may first image the edible crowns, and then if ready forharvesting, may harvest the edible crowns using the robotic arms. Thisallows the computing system to instruct the robotic arms to harvest theedible crowns. Mounting or fixing the imaging systems in this manneralso allows the harvester to continuously move across the field andcontinuously determine whether the edible crowns are ready forharvesting.

The robotic arms may function to pick the edible crowns and separate theedible crowns from other portions of the broccoli plant (e.g., stem). Insome instances, the harvester may include any number of robotic armshaving the end effectors for harvesting the edible crowns. For example,the harvester may include one or more robotic arms for each row ofbroccoli plants being harvested (or imaged). In some instances, theharvester may be configured to harvest multiple rows broccoli plants atthe same time, using the one or more robotic arms. In this sense, therobotic arms may be configured to harvest individual edible crowns frombroccoli plants of a particular row, or the harvester may include anynumber of robotic arms for harvesting the edible crowns from any numberof rows. For example, in instances where the harvester is configured toharvest twelve rows of broccoli plants, the harvester may include afirst robotic arm for harvesting the broccoli plants within a first row,a second robotic arm for harvesting the broccoli plants within a secondrow, and so forth. However, in some instances, the robotic arms mayharvest across rows of broccoli plants. For example, a particularrobotic arm may harvest edible crowns within the first row and thesecond row, or two robotic arms may harvest edible crowns across thefirst row and the second row.

Each robotic arm may include a mechanical device for picking the ediblecrowns and cutting the edible crown from the stem. For example, the endeffector of the robotic arm may represent a gripper that is moveablebetween an open state and a closed state. The end effector may includelimbs, members, or fingers that enclose around the edible crown in theclosed state, and which are sized to receive the edible crown withoutbruising or otherwise damaging the edible crown. In some instances,actuators may open and close the fingers of the end effector, betweenthe open state and the closed state. When open, the fingers may bespaced apart by distances that allows the end effector to descend overor upon the edible crown. The end effector, via fingers, converge uponand grasp the broccoli stem in the closed state. When closed, the endeffector may include an interior space or cavity occupied by the ediblecrown.

The fingers may include shapes and sidewalls for receiving andsupporting the edible crown once cut from the broccoli stem. The endeffector may include a cutting mechanism (e.g., blade, saw, knife, etc.)for severing the edible crown from the broccoli stem. In some instances,the cutting mechanism may represent an actuatable rotary blade thatrotationally cuts through the broccoli stem, or may represent a singleguillotine blade and/or double guillotine blade that linearly cutsthrough the broccoli stem. After the edible crown is cut from thebroccoli stem, the edible crown may remain cradled in the end effector,within the fingers, for transporting to other portions of the harvesterfor processing and/or collection. In some instances, the end effectormay include two, four, or any number of fingers. Additionally, in someinstances, once the cutting mechanism cuts through the broccoli stem,the cutting mechanism (i.e., the blade) may remain in position tosupport the edible crown in the end effector. For example, the cutbroccoli stalk may rest on the blade to support the edible crown withinan interior of the end effector.

When the computing system classifies, tags, or otherwise identifies anedible crown as being ready for harvesting, the robot arm may maneuveror position to the edible crown. In some instances, positioning therobotic arm may involve moving the end effector over (e.g., above) theedible crown. In some instances, the robotic arm may be configured tomove the end effector along, or in, multiple planes. In some instances,the robotic arm and/or the end effector may be coupled to thepositioning system (e.g., tracks rails, motors, etc.) that effectuatesto position the end effector relative to the edible crown. The endeffector may therefore have multiple degrees of freedom to accommodatefor the varying characteristic(s) of the broccoli plants. For example,as broccoli plants often do not grow in straight lines and are also notalways vertical, the end effector may be moved to be centered orpositioned relative to the edible crown being harvested. The ediblecrowns may also not stand at a uniform height above the ground, makingit impractical to cut the broccoli stems at a given height above theground. The robotic arm may therefore include actuators that areconfigured to position the end effector for harvesting the edible crown.

In some instances, the harvester may utilize the imaging system forpositioning the end effector. For example, in addition to using theimaging system to detect harvestable edible crowns, the image(s) mayalso be used to determine a central point of the broccoli plant (or ofthe edible crown). For example, the image(s) may be analyzed todetermine a size of the edible crown (e.g., diameter, height, volume,etc.), and correspondingly the central point of the edible crown withincoordinate space. This central point may be used, at least in part, forpositioning the end effector.

In some instances, the computing system may determine a rangecoordinates associated with the edible crown. The range of coordinates(X and Y) may represent an area within coordinate space. This range ofcoordinates may correspond to the edible crown, and from this range, thecentral point of the edible crown along one or more axes may bedetermined. For example, the central point may be represented in X and Ycoordinate space and may be used to position the end effector above theedible crown. Furthermore, as part of analyzing the image(s), thecomputing system may determine a depth (Z-direction) of the ediblecrown. This depth may be used when instructing the end effector todescend upon the edible crown (e.g., towards the ground).

In some instances, the central point may be determined for eachharvestable edible crown and the central point may be utilized foraligning the end effector along multiple planes (or axes). For example,the central point may be a center of the edible crown (e.g., center ofmass, center of area/volume, etc.). In some instances, the central pointmay first be used to position the end effector vertically above theedible crown, and then second, to lower the end effector towards theedible crown to converge upon the edible crown. In some instances, theend effector may be lowered to a certain position for cutting a certainamount of broccoli stalk. For example, depending on consumerpreferences, the harvested edible crown may include different lengths ofstalks attached thereto.

To illustrate, after determining a harvestable edible crown, thecomputing system may determine that the center point of the edible crownis located at X₁, Y₁, and Z₁ in coordinate space. The computing systemmay instruct the robotic arm (or other components) to move the endeffector (or move the robotic arm) to align the end effector along afirst horizontal axis/plane (e.g., X-axis/X-plane) of the edible crowncorresponding to X₁. In this position, the end effector may be alignedwith the center point of the edible crown in a first direction.Subsequently, or continuously, the computing system may instruct therobot arm to move the end effector to align the end effector along asecond horizontal axis/plane (e.g., Y-axis/Y-plane) of the edible crowncorresponding to Y₁. In this position, the end effector may be alignedwith the center point of the edible crown in the first direction and asecond direction. That is, once positioned above the edible crown, atX₁, Y₁, the end effector may be substantially concentric or aligned withthe center of the edible crown along two axes, or planes (e.g.,X-axis/X-plane and Y-axis/Y-plane).

Once positioned above the edible crown, the computing system mayinstruct the robot arm to descend the end effector, in the open state,upon the edible crown. In some instances, the end effector may descendby a distance (in the Z-direction) such that once the end effector isclosed around the edible crown, the end effector (or the fingers of theend effector) grasps the broccoli stem at a position under the ediblecrown. In some instances, the end effector may descend a predetermineddistance relative to the center point of the edible crown, or relativeto the Z₁ coordinate of the edible crown. Once the end effector descendsupon the edible crown, the fingers may be actuated by the robotic arm toenclose the edible crown.

Once grasped, the cutting mechanism may cut the edible crown from thebroccoli stem. For example, the cutting mechanism may include arotatable blade or blade that linearly cuts through the broccoli stem.In some instances, the cutting mechanism may cut the broccoli stem at aposition below (e.g., towards the ground) where the fingers grasp thebroccoli stem (e.g., outside the cavity of the end effector in which theedible crown resides). The cutting mechanism therefore severs the ediblecrown from the broccoli stem while the edible crown is retained withinthe end effector. Noted above, the amount of broccoli stalk (or stem)left attached to the harvested edible crown may be varied and/orprogrammable. When the broccoli stem of the encapsulated edible crownhas been cut, the edible crown is retained within the end effector andtransported for discharge to one or more collection points. For example,the actuators that are utilized to position the robotic arm (or the endeffector) may be used to maneuver the robotic arm to a collection pointwhereby the end effector transitions to the open state and deposits theharvested edible crown.

In some instances, the end effector, and particularly the fingers, maybe utilized to strip leaves from the broccoli stems. For example, oncethe end effector grasps the broccoli stem and before the cuttingmechanism cuts the broccoli stem, the robotic arm may descend the endeffector downward towards the ground (away from the edible crown). Bydescending the end effector downward, with the gripper in the closedstate, or partially closed state, the fingers may scrape or traversealong the broccoli stem. In some instances, this movement may cause theend effector (or the fingers) to strip the leaves away from the ediblecrown to dispose of foliage adjacent the edible crown. For example, thefingers, such as sidewalls thereof, may deflect or displace the leaves(or other foliage) away from the edible crown and effectuate to breakthe leaves away from or strip the leaves from the broccoli stem. Afterdescending a predetermined distance, the end effector may ascend upwardbefore reaching a position whereby the cutting mechanism cuts thebroccoli stem below the edible crown. In some instances, the endeffector may additionally or alternatively strip the leaves while thefingers are still in the open state but moving towards the closed state.

As noted above, after being cut, the edible crown may be moved to one ormore collection points. For example, the robotic arm may move the ediblecrown to a collection point for unloading the edible crown from the endeffector. In some instances, the one or more collection points mayrepresent a bin, or other container, configured to receive ediblecrowns. Additionally, or alternatively, the one or more collectionpoints may be a conveyor belt for transporting the harvested ediblecrowns to other portions of the harvester. Regardless, at the one ormore collection points, the end effector transitions to the open stateand the fingers are opened to release the edible crowns.

In some instances, the use of the end effector, including the fingersand the cutting mechanism, may contribute to the efficiency of theharvester. For example, integrally forming the cutting mechanism withinthe end effector may reduce a number of components, and lead to anincrease or accuracy in severing the edible crown from the broccolistem. Fewer parts for the end effector, as compared to existingharvesters, may also reduce maintenance to keep the end effector inproper working condition.

As discussed above, the harvester may continuously move across the fieldand the robotic arms may continuously harvest the edible crowns whilethe harvester is in motion. In some instances, to account for themovement of the harvester, the robotic arms may be configured to move inrelation, or relative, to the harvester. For example, the robotic armmay move in relation to the harvester and based on the speed of theharvester to account for the movement of the harvester across the field.This may, in some instances, include the robotic arm moving in adirection different from (e.g., opposite) a direction of travel of theharvester to remain centered above the edible crown. As the harvestertravels along the rows, the robot arms may maintain the position of theend effector relative to the edible crown to compensate for thecontinuous movement of the harvester.

In some instances, the harvester may perform different types or multipletypes of cuts based on characteristic(s) of the broccoli plant. Forexample, certain types of cuts may be performed based on a diameter (orlargest cross-sectional dimensions) of the edible crown. For example, afirst type of cut may be performed if the diameter of the edible crownis within a first range of sizes, such as 4.0 and 4.75 inches. The firsttype of cut may involve the harvester, via the end effector and thecutting mechanism, only cutting the edible crown from the broccoli stem.Alternatively, a second type of cut may be performed if the diameter ofthe edible crown is within a second range of sizes, such as 4.75 and5.75 inches. The second type of cut may involve the harvester, via theend effector and the cutting mechanism, stripping the leaves around theedible crown and then cutting the edible crown from the broccoli stem.In some instances, the second type of cut may be performed for ediblecrowns having a diameter over 4.75 inches. As such, the harvester mayperform different operations for harvesting the edible crown based on adiameter (or other characteristic(s)) of the edible crown.

Although the discussion herein relates to harvesting broccoli, orprocesses of harvesting broccoli, the harvester may be utilized toharvest other crops, such as other standing or stalk-based vegetablecrops (e.g., cauliflower, asparagus, celery, lettuce, etc.). In suchinstances, the harvester or portions thereof may be modified to handlelarger or differently shaped plants, and a ML model(s) may be trained onimages of these other types of crops for selective harvesting.Accordingly, it is to be appreciated that the term “broccoli” may beinterchanged with other types of crops (or plants) throughout thisdisclosure.

The present disclosure provides an overall understanding of theprinciples of the structure, function, device, and system disclosedherein. One or more examples of the present disclosure are illustratedin the accompanying drawings. Those of ordinary skill in the art willunderstand and appreciate that the devices, the systems, and/or themethods specifically described herein and illustrated in theaccompanying drawings are non-limiting embodiments. The featuresillustrated or described in connection with one embodiment, or instance,may be combined with the features of other embodiments or instances.Such modifications and variations are intended to be included within thescope of the disclosure and appended claims.

FIG. 1 illustrates an example harvester 100 for harvesting broccoli. Theharvester 100 may be configured to operate within a field 102 containinga plurality of broccoli plants 104 that are grown in rows. As theharvester 100 traverses or moves across the field 102, the harvester 100functions to harvest the broccoli plants 104. As shown, and as discussedherein, the harvester 100 includes components to harvest the broccoliplants 104 across multiple rows.

The harvester 100 includes a frame 106 that supports components of theharvester 100 or to which components of the harvester 100 mount, couple,or are disposed. The frame 106 may, in some instances, comprise a bodyand provide a platform for supporting the components of the harvester100, as will be discussed herein.

The harvester 100 includes wheels 108 for elevating the frame 106 abovethe field 102 and the broccoli plants 104, and for moving the harvester100 about the field 102. In some instances, the wheels 108 may bedisposed at each corner (or substantially at each corner) of theharvester 100 (or the frame 106). Additionally, or alternatively, thewheels 108 may be disposed on opposing ends or sides of the harvester100. For example, in some instances, the harvester 100 may include afirst side 110 and a second side 112, spaced apart from the first side110 in the Y-direction. The first side 110 may include two of the wheels108 that couple or mount to a first side of the frame 106 and the secondside 112 may include two of the wheels 108 that couple or mount to asecond side of the frame 106. However, in some instances, the harvester100 may include less than or more than four wheels or the wheels 108 maybe located on the harvester 100 differently than shown. For example, thewheels 108 may be spaced farther apart or may be spaced closer together.Additionally, or alternatively, the harvester 100 may include continuoustracks (e.g., rubber), or a track system, for driving the harvester 100.

The wheels 108 may be spaced apart or offset from one another such thatthe wheels 108 are positioned in between rows of the broccoli plants104. For example, as shown, each of the rows of the broccoli plants 104may be separated by a predetermined distance (e.g., in the X-direction).This predetermined distance may be determined or known during plantingof the broccoli plants 104. By way of example, the predetermineddistance may be eight inches, ten inches, twelve inches, or any otherdistance. Interposed between the rows, the broccoli plants 104 are notplanted, and hence, driving the wheels 108 within this gap does notdamage the broccoli plants 104 or reduce a harvestable yield of thebroccoli plants 104. Therefore, the size of the wheels 108 and/or thedistance at which the wheels 108 are spaced apart from one another mayaccommodate the size of the broccoli plants 104 as well as the spacingin between the rows of broccoli plants 104.

The wheels 108 may operably couple to a driving mechanism of theharvester 100, such as a motor or engine (e.g., combustion and/orelectrical). Additionally, or alternatively, the harvester 100 may besolar-powered, battery powered, and/or a combination thereof. The motorand the coupling of the motor to the wheels (e.g., transmission,differential, gearbox, linkages, pneumatics, etc.) forms a drivetrainthat powers the harvester 100 across the field 102. In some instances,the harvester 100 includes a centralized motor that powers all of thewheels 108. However, in some instances, each of the wheels 108 mayinclude their own electrical motor that is powered by a generator 114 ofthe harvester 100. The wheels 108 may therefore be independentlyactuatable by a respective motor.

In some instances, the individual motors may be located on a hub of theindividual wheels 108, and may receive power from the generator 114 forpowering the wheels 108. The individual motors may power each of thewheels 108 at respective speeds. Additionally, the wheels 108 may beconfigured to be powered in one or more directions (e.g., clockwise andcounterclockwise, forward and reverse, etc.) for directing the harvester100 forward and backwards. Independently powering each of the wheels 108may also increase a maneuverability of the harvester (e.g., turningradius) to make minor adjustments in steering the harvester 100 withinthe field 102 and directing the harvester 100 along a predeterminedroute within the field 102.

Additionally, or alternatively, each of the wheels 108 may be steerable.In some instances, the wheels 108 may be rotated 180 or 360 degrees(about Z-axis). For example, in the case that the harvester 100 includesfour wheels, each of the wheels 108 may be steerable (e.g., four-wheelsteering). The independent steering of the wheels 108, as well asindependently powering each of the wheels 108, allows the harvester 100to turn or position across the field 102 in real-time and make minoradjustments to positioning. This sharp turning increases amaneuverability of the harvester 100.

In some instances, the harvester 100 may be controlled or operated via acabin or cockpit 116. The cockpit 116 includes an operator who operatesand controls the harvester 100. For example, within the cockpit 116, theoperator may steer the harvester 100 or may control a speed and/ordirection of the harvester 100 within the field 102. The operator mayalso control an amount of power supplied to each of the wheels 108 forsteering the harvester 100. Within the cockpit 116, the operator mayalso utilize various instruments, gadgets, panels, and so forth forcontrolling the operation of the harvester 100. For example, a panelwithin the cockpit 116 may illustrate a route for harvesting thebroccoli plants 104, an amount of the broccoli plants 104 that are beingharvested, a status of the harvester 100, and so forth.

As will be discussed herein, the harvester 100 may travel in more thanone direction for harvesting the broccoli plants 104. For example, thewheels 108 may rotate clockwise to propel the harvester 100 in a firstdirection of travel (e.g., as shown in FIG. 1), and subsequently, thewheels 108 may rotate counterclockwise to propel the harvester in asecond, opposite direction. Alternatively, the wheels 108 may be poweredin an opposite direction without rotating the wheels 108. To accommodatefor this multi-directional movement, a seat within the cockpit 116 maybe adjustable to swivel and the operator may face a respective directionof travel. In such instances, the cockpit 116 may include more than oneset of panels, gadgets, instruments, and/or a steering wheel forcontrolling the harvester 100.

FIG. 1 further illustrates, that in some instances, skirts 118 may beplaced around the wheels 108. The skirts 118, in some instances, mayprevent build-up of dirt, mud, or other debris on the wheels 108. Forexample, the skirts 118 may come into close proximity with a sidewalland/or outer periphery of the wheel 108 (or a tire thereof) to scrape ordeflect mud build-up. As discussed herein, in some instances, the wheels108 or portions of the harvester 100 may include an encoder for trackinga distance the harvester 100 travels. This distance may be used todetermine a position of the harvester 100 within the field 102 and/or aposition of a harvestable edible crown within the field 102. However,the build-up of mud or other debris may impact an accuracy indetermining a distance traveled by the harvester 100 (e.g., slippage,diameter of the wheel 108, etc.). As such, the skirts 118 may functionto reduce this build-up, which may be used to accurately determine aposition of the harvester 100 and/or the edible crowns ready forharvesting. However, as also discussed herein, other positioning systems(e.g., Global Positioning Satellite (GPS)) may be used for trackingand/or determining a location of the harvester 100 within the field 102.

The harvester 100 may include an assembly or a hood 120 that extendsfrom the frame 106. Further details of the hood 120 are discussedherein. Generally, the hood 120 functions to image the broccoli plants104 and if the broccoli plants 104 are ready for harvesting (e.g.,ripe), components of the harvester 100 pick or harvest the individualbroccoli plants 104. The hood 120 is shown extending from the frame 106,or being supported by the frame 106, and disposed above a groundsurface.

In some instances, the hood 120 may couple to the frame 106, or otherportions of the harvester 100, using pneumatic cylinders to raise andlower the hood 120 relative to the field 102 and/or the broccoli plants104. For example, when not in use, the hood 120 may be raised(Z-direction) so as to be in closer proximity to the frame 106 than asshown in FIG. 1, elevated above the field 102. However, in someinstances, the hood 120 may rigidly couple to the frame 106.

In some instances, the hood 120 may include a rectangular shape. Thehood 120 is shown being disposed between the wheels 108 of the harvester100 on the first side 110. A gap distance may be disposed between thewheels 108 and ends of the hood 120 to allow the wheels 108 to rotate(e.g., about the Z-axis) without contacting or abutting the hood 120.

As shown, the broccoli plants 104 may pass underneath the hood 120 asthe harvester 100 moves in the direction of travel. As shown in FIG. 1,six rows of the broccoli plants 104 are configured to pass underneaththe hood 120 at a given time. The harvester 100 may therefore beconfigured to harvest six rows of the broccoli plants 104 simultaneouslyor at the same time. However, the harvester 100 may be scaled to harvestmore than or less than six rows of the broccoli plants 104 at a singletime or instance. For example, the harvester 100 may include componentsfor harvesting twelve rows of broccoli plants 104 at the same time. Insuch instances, the harvester 100 may include components and/or featuresto account for the increased harvesting, such as a lengthened frame, oneor more additional wheels, and so forth.

Furthermore, although FIG. 1 illustrates a particular arrangement of therows of the broccoli plants 104, the harvester 100 may be configured toharvest rows of the broccoli plants 104 that have differentcharacteristics. For example, the broccoli plants 104 may not be plantedin a straight line (e.g., zig-zag, curved, etc.), the rows of thebroccoli plants 104 may be spaced closer together, the rows of thebroccoli plants 104 may be spaced farther apart, the rows of thebroccoli plants 104 may not be evenly spaced apart, the individualbroccoli plants 104 may not be evenly spaced apart from one anotherwithin the same row, or in some instances, multiple rows of the broccoliplants 104 may be planted in close proximity to one another and spacedapart by a greater distance from other rows. For example, two rows ofthe broccoli plants 104 may be planted in close proximity, with minimalspacing therebetween, and these two rows may be spaced apart from anadditional two rows. To accommodate for these variables and changingcharacteristics, as will be discussed herein, the harvester 100 may beconfigured to harvest a multitude of the broccoli plants 104 accordingto their arrangement and planting within the field 102.

The hood 120 is shown being disposed vertically above the broccoliplants 104 (Z-direction) for imaging the broccoli plants 104 as thebroccoli plants 104 pass underneath the hood 120 and as the harvester100 moves in the direction of travel. Generally, the broccoli plants 104include a stalk 122 growing upwards from the ground and buds that growon an end thereof, above the ground. The buds form an edible crown 124that is harvested for consumption. The edible crown 124 may correspondto a portion of the broccoli plants 104 that are harvested forconsumption. In some instances, the edible crown 124 may be referred toas a head of the broccoli plant 104, a floret of the broccoli plant 104,a flower of the broccoli plant, or an edible portion of the broccoliplant 104.

The edible crown 124 of the individual broccoli plants 104 may be imagedand this imaging, or the image(s) generated by imaging device(s) and/orsystem(s) of the harvester 100, may be utilized to determine whether toharvest the broccoli plants 104. For example, image analysis and/or MLmodel(s) may be used to determine whether the broccoli plants 104 areready for harvesting (e.g., ripe, mature, etc.). If so, components ofthe harvester 100 may harvest the broccoli plants 104 (e.g., the ediblecrowns 124). For example, and in some instances, the harvester 100 mayinclude robotic arms having actuatable end effectors and/or cuttingmechanisms that harvest the broccoli plants 104.

By way of example, and in instances where the harvester 100 isconfigured to harvest six rows of the broccoli plants 104simultaneously, the harvester 100 may include six imaging devices (e.g.,cameras and/or IR sensors) and six robotic arms having the end effectorsand/or cutting mechanisms. The six imaging devices may image arespective row of the broccoli plants 104, while the six end effectorsof the six robotic arms may pick the broccoli plants 104, respectively,and the six cutting mechanisms of the six end effectors may cut(harvest) respective broccoli plants 104. The robotic arms may beindependently actuatable and controlled for harvesting the edible crowns124 within the respective rows of broccoli plants 104. However, in someis instances, the imaging devices may image the broccoli plants 104across one or more rows and/or or the end effectors may pick the ediblecrowns 124 across one or more row of the broccoli plants 104.

In some instances, the imaging devices may be secured within the hood120, or on a bottom surface of the hood 120. In some instances, theimaging devices may be stationary or may be movable or actuatable in oneor more directions (e.g., X-direction, Y-direction, Z-direction) and/orone or more degrees of rotation (e.g., tilt, pan, yaw) for capturing theimage(s) of the broccoli plants 104.

As noted above, robotic arms of the harvester 100 may harvest the ediblecrowns 124 and/or include components for harvesting the edible crowns124. In some instances, the robotic arms may include actuators, asuspension system, or members that are configured to position the endeffectors relative to the broccoli plants 104. For example, FIG. 1illustrates one or more robotic arms 126 that extend from the frame 106,downward (Z-direction), and which couple to respective end effectors.The robotic arms 126, additionally or alternatively, may extend fromother portions of the harvester 100, such as the hood 120. In someinstances, a first end of the robotic arms 126 may couple to the frame106 (or the hood 120), while a second end of the robotic arms 126 maycouple to the end effector for positioning the end effector relative tothe individual broccoli plants 104 that are ready for harvesting. Inthis sense, the robotic arms 126 may articulate or position to locatethe end effectors.

The end effectors are configured to harvest the broccoli plants 104, andspecifically, the edible crown 124 of the broccoli plants 104. Theedible crown 124 may be grasped by the end effector, cut from the stalk122, and transferred to a collection location on the harvester 100. Forexample, in some instances, the end effectors may transfer the harvestededible crowns 124 to a conveyor belt or other transfer mechanism, (e.g.,flipper, chute, escalator, etc.) that transfers the harvested ediblecrowns 124 to other portions of the harvester 100.

In some instances, the harvester 100 may include more than one hood 120,the harvester 100 may be designed to travel in multiple directions. Forexample, the second side 112 of the harvester 100 may include a secondhood that includes similar components as the hood 120 and/or the secondside 112 of the harvester 100 may include similar components as thefirst side 110. Including a second hood may increase a harvesting yieldof the broccoli plants 104, increase a universalness of the harvester100, and/or increase a rate of harvesting. For example, the second hoodon the second side 112 may be utilized when the harvester 100 travels ina second direction, opposite the direction of travel as shown in FIG. 1.For instance, upon the harvester 100 reaching the end of the field 102or the rows of the broccoli plants 104, the harvester 100 may utilizethe second hood for imagining the broccoli plants 104 in lieu of thehood 120. This may avoid the harvester 100 having to perform wide turnsat the end of the field 102. Instead, the harvester 100 may steer thewheels 108 to additional rows (e.g., by driving the harvester 100 in theX-direction of FIG. 1), without having to perform wide “U-turns” andthen travel in a direction opposite the direction of travel. In suchinstances, the second hood may be used to image broccoli plants 104within additional rows and the robotic arms 126 may pick and harvestthese broccoli plants 104. The harvester 100 may therefore travel inmultiple directions for harvesting the broccoli plants 104, where theharvester 100 may include the robotic arms 126 that are configured tooperate in conjunction with the imaging performed by the harvester 100,whether by the hood 120 and/or an additional hood.

The harvester 100 is shown including a platform 128 on which personnelstand. The personnel may perform further processing on the harvestededible crowns, such as cleaning, removing leaves, sorting (e.g., size,color, shape, maturity, etc.), discarding, repurposing, and so forth.For example, after the edible crowns are harvested, transfer mechanisms(e.g., conveyor belt, ladder, escalator, lift, etc.) may transfer theedible crowns to the platform 128. In some instances, the transfermechanisms may transfer the edible crowns to conveyor belts 130 on theplatform 128. For example, FIG. 1 illustrates that the platform 128 mayinclude two conveyor belts 130. A first of the conveyor belts 130 may beoperated by a first portion of the personnel, while a second of theconveyor belts 130 may be operated by a second portion of the personnel.As the conveyor belts 130 operate, edible crowns may pass along theconveyor belts 130 and the personnel may inspect the edible crowns.Additionally, or alternatively, the personnel may box or package theedible crowns for shipment or distribution.

The frame 106, or the platform 128, may further include guardrails forsafety, a canopy to provide shade and/or shelter for the personnel, binsor cabinets for storing supplies (e.g., boxes, gloves, etc.), and/orladders for allowing the platform 128 to be entered and exited.

The harvester 100 is therefore configured to operate while traversingacross the field 102 without stopping. For example, as the wheels 108power and/or steer the harvester 100, the broccoli plants 104 are imagedby the imaging devices for use in determining whether to harvest theedible crowns 124. Subsequently, the robotic arms 126 may pick andharvest the edible crowns 124 that are ready. The harvester 100 maytherefore continuously image the edible crowns 124 and pick the ediblecrowns 124 that are ready for harvesting, all while the harvester 100moves about this field 102. Such continuous movement (e.g., half a mileper hour, one mile per hour, two miles per hour, etc.) may increaseyields and reduce harvesting times. Moreover, the harvester 100 may beconfigured to travel at respective speeds based on a density ofharvestable edible crowns or the number of rows the harvester 100 isconfigured to harvest.

Furthermore, although the discussion herein relates to harvesting ediblecrowns 124 of the broccoli plants 104, the harvester 100 may beconfigured or utilized to harvest other crops, such as lettuce,cauliflower, asparagus, brussels sprouts, and so forth. In suchinstances, the harvester 100 may be equipped with suitable equipment,imaging devices, robotic arms (pickers), end effectors, and so forth.

In some instances, although the harvester 100 is discussed herein asbeing a self-propelled machine, the harvester 100 may be configured tobe towed, pulled, or carried by a tractor, for example. In suchinstances, the components of the harvester 100 may be powered and/ordriven by components of the tractor. For example, components of theharvester 100 may be driven by hydraulic motors powered by a hydraulicpump driven from a power take off (PTO) of the tractor. Additionally, oralternatively, electrical components within the harvester 100 may bepowered from an electrical system of the tractor, or via onboardgenerator of the harvester 100.

FIG. 2 illustrates an additional view of the harvester 100. For example,FIG. 1 illustrated the first side 110 of the harvester 100, moving inthe direction of travel, while FIG. 2 illustrates the second side 112 ofthe harvester 100 while the harvester 100 is moving in the direction oftravel. In this sense, the view illustrated in FIG. 2 may illustrate aback or trailing side, relative to FIG. 1, as the harvester 100 moves inthe direction of travel.

As discussed above, the harvester 100 may include the wheels 108 locatedon the second side 112 and/or an additional hood 200. For example, asthe harvester 100 travels in an opposite direction of travel (oppositeto the direction of travel shown in FIGS. 1 and 2), the additional hood200 may be used for harvesting the broccoli plants 104. For example,when the harvester 100 is moving in the direction of travel as shown inFIG. 2, the additional hood 200 may not be in use for imaging and/orotherwise harvesting the broccoli plants 104. In some instances, theadditional hood 200 may be raised to an elevated position when not inuse using pneumatic cylinders and/or arms. In some instances, as theharvester 100 travels in the indicated direction of travel(Y-direction), the additional hood 200 may be on a trailing side,opposite a leading side, where the hood 120 is located. In someinstances, the additional hood 200 on the trailing side may be used inconjunction with the hood 120 on the leading side for redundancypurposes (e.g., for imaging edible crowns 124 and/or harvesting ediblecrowns 124 that might have been missed by the hood 120) or to imagethose edible crowns 124 that were not harvested and/or which were notready for harvesting.

FIG. 2 also illustrates that as the harvester 100 picks the ediblecrowns 124 from the broccoli plants 104, and after the personnelprocesses or boxes the edible crowns 124, packages 202 may betransferred to an awaiting vehicle 204 (or trailer). In some instances,the conveyor belts 130 may carry or transfer the packages to personnelon or operating the vehicle 204. The personnel may stack the packages ona bed of the vehicle 204 for shipment.

In some instances, the packages 202 may be transferred to the vehicle204 while the harvester 100 is moving, or the harvester 100 may park tooffload the packages 202. However, transferring the packages 202 to thevehicle 204 while the harvester 100 is harvesting the edible crowns 124may permit uninterrupted harvesting of the edible crowns 124.

As shown in FIG. 2, some of the broccoli plants 104 may not beharvested. For example, as the imaging devices image the broccoli plants104, some of the edible crowns may not be ready for harvesting. By wayof example, the edible crowns 124 may be of insufficient size, color,shape, density, or maturity. If the edible crowns 124 are not ready forharvesting (e.g., not mature, not ripe, etc.), the harvester 100 may notharvest these edible crowns and as such, unharvested broccoli plants 206(of the harvested broccoli plants 104) may remain planted in the field102. In other words, the robotic arms 126 may not be instructed to pickcertain edible crowns of the broccoli plants 104 (e.g., those ediblecrowns corresponding to the unharvested broccoli plants 206). However,the robotic arms 126 are instructed to harvest those edible crowns ofthe broccoli plants 104 that are ready for harvesting.

The unharvested broccoli plants 206 may therefore remain planted forharvesting at a later instance. In some instances, the harvester 100 mayrecord a location of the unharvested broccoli plants 206. This recordedlocation may be used at a later instance to locate the unharvestedbroccoli plants 206 within the field 102. In other instances, theharvester 100 may record characteristic(s) of the unharvested broccoliplants 206 that are not harvested (e.g., size, color, shape, etc.) andmay project, based on these characteristic(s), when the edible crowns ofthe unharvested broccoli plants 206 will be ready for harvesting. Suchdetermination may be used at a later instance for harvesting theunharvested broccoli plants 206. Additionally, or alternatively, imagesof the unharvested broccoli plants 206 may be used to train (orre-train) ML model(s). For example, knowing characteristic(s) of theunharvested broccoli plants 206 may increase an accuracy of the MLmodel(s) determining or recognizing those broccoli plants that are readyfor harvesting and/or not ready for harvesting.

Additionally, or alternatively, in some instances, the additional hood200 may determine, among the broccoli plants 104, the unharvestedbroccoli plants 206. For example, image(s) captured by the imagingdevices of the additional hood 200 may be used to determine a locationof the unharvested broccoli plants 206 and/or characteristics of theunharvested broccoli plants 206, for use in determining when theunharvested broccoli plants 206 will be ready for harvesting.

FIG. 3 illustrates a side view of the harvester 100. As shown, theharvester 100 travels in the direction of travel for harvesting certainedible crowns 124 of the broccoli plants 104.

As discussed previously, the harvester 100 may include two hoods forimaging the edible crowns 124. For example, the harvester 100 mayutilize a first hood 300 (e.g., the hood 120) as the harvester 100travels in the direction of travel for imaging the edible crowns 124. Asecond hood 302 (e.g., the additional hood 200) may be utilized by theharvester 100 when traveling in a direction opposite to the direction oftravel. In this sense, the harvester 100 may not include a designated“front” or “back” but may travel in multiple forward directions.

Between the first hood 300 and the second hood 302, or between thewheels 108 on the first side 110 and the wheels 108 on the second side112 may be an internal space occupied by the robotic arms 126. Therobotic arms 126, as discussed above, may descend from the frame 106 orother portions of the harvester 100 for harvesting the edible crowns 124as the broccoli plants 104 pass under the first hood 300 and are flaggedfor harvesting.

The wheels 108 of the harvester 100 are further shown being turned atvarious angles or orientations to navigate the harvester 100 throughoutthe field 102.

FIG. 4 illustrates a side view of the harvester 100, such as the firstside 110. The hood 120 is shown being disposed vertically above thebroccoli plants 104 (e.g., Z-direction) such that the imaging devicesmay image the edible crowns 124 for use in determining whether theedible crowns 124 are ready for harvesting. Discussed above in relationto FIG. 3, positioned behind the hood 120 (Y-direction) may be therobotic arms 126 having the end effectors that harvest the edible crowns124 of the broccoli plants 104. After the end effectors harvest theedible crowns 124, those edible crowns may be transferred to theplatform 128 for processing (e.g., packaging) by the personnel.

FIG. 4 illustrates that the hood 120 is disposed vertically above thebroccoli plants 104, with a certain distance interposed between a top ofthe broccoli plants 104 and a bottom 400 of the hood 120. In someinstance, the hood 120 may be disposed a predetermined distance abovethe ground, or above the top of the broccoli plants 104 (or of theedible crowns 124). In some instances, the harvester 100 may optionallyinclude a sensor for determining a distance between the bottom 400 ofthe hood 120 and the ground (or a distance between the bottom 400 andthe top of the edible crowns 124), and correspondingly, causing a heightof the hood 120 to be adjusted (Z-direction).

As illustrated, the hood 120 may include a first end 402 and a secondend 404 disposed between the wheels 108 on the first side 110. In someinstances, the harvester 100 may include support wheel 406 at the secondend 404 for supporting a weight of the hood 120 and/or adjusting anelevation of the hood 120 above the ground. Additionally, oralternatively, the harvester 100 may include a support wheel at thefirst end 402 of the hood 120. As the harvester 100 moves across thefield 102, the support wheel 406 may traverse across the ground foradjusting the hood 120 upward and/or downward (Z-direction). Forexample, when the support wheel 406 experiences an uphill movement, alinkage connected with the hood 120 may push the hood 120 upwards, awayfrom the ground. This may prevent the hood 120 running into or hittingthe edible crowns 124. Alternatively, when the support wheel 406experiences a downhill movement, the linkage connected to the hood 120may pull the hood 120, closer to the ground and towards the ediblecrowns 124. Positioning the hood 120 closer to the ground may result inthe imaging devices being closer to the broccoli plants 104, which mayincrease an image quality of the edible crowns 124.

FIG. 4 also illustrates a different arrangement of the broccoli plants104. As shown, and in some instances, the broccoli plants 104 may beplanted in rows of two, which are spaced apart from an adjacent two rowsof the broccoli plants 104. In some instances, an imaging device mayinclude a first camera for imaging a first of the two rows of broccoliplants 104, while a second camera of the imaging device may image asecond of the two rows of broccoli plants 104. Each of the rows, mayinclude a respective robotic arm (including the end effector and/orcutting mechanism), or the harvester 100 may include a single roboticarm for harvesting the two rows of the broccoli plants 104.

Additionally, FIG. 4 illustrates forks that extend from the frame 106for supporting and coupling to the individual wheels 108. Motors 408 maybe coupled to the axle of the wheel 108 for powering the harvester 100across the field 102.

FIG. 5 illustrates selected components of the harvester 100. Thecomponents listed and discussed are merely exemplary and it is to beunderstood that the harvester 100 may include additional and/ordifferent components for carrying out the operations described hereinand for harvesting edible crowns.

The harvester 100 may include a computing system 500 that functions tocarry out perform the described operations, as well as controllingcomponents of the harvester 100. The computing system 500 is shownincluding processor(s) 502 and computer-readable media 504. Theprocessor(s) 502 may perform various operations described herein. Asshown, the computer-readable media 504 may store or otherwise haveaccess to various information, including instructions that, whenexecuted, cause the processor(s) 502 to perform the operations describedherein. The processor(s) 502 also communicatively couple to componentsof the harvester 100 for receiving data and transmitting instruction, aswell as causing data to be stored within the computer-readable media504.

As discussed above, the harvester 100 may include components formaneuvering about the field 102, such as a driving mechanism 506, anavigational system 508, and/or a tracking system 510. In someinstances, the driving mechanism 506 may include components for poweringthe harvester 100, such as motors and/or engines (combustion orelectric), batteries, solar panels, components for driving the harvester100, such as wheels (e.g., the wheels 108) or continuous tracks,components for distributing and/or transfer the power throughout theharvester 100 (e.g., transfer cases, differentials, gear boxes,electrical boxes/cables/lines), and components for directing theharvester 100, such as steering devices. As discussed above, each of thewheels 108 may be independently powered and/or steered. The drivingmechanism 506 may also include the generator 114, which may power thewheels 108 and/or other components of the harvester 100.

The navigational system 508 may include components for navigating theharvester 100 within the field 102. For example, the navigational system508 may include a global position system (GPS) that is utilized tonavigate the harvester 100 along routes or certain paths throughout thefield 102. The navigational system 508 may, in some instances, beutilized by the operator of the harvester 100 for steering the harvester100. In some instances, the navigational system 508 may control ortransmit instructions to the driving mechanism 506, or componentsthereof. The instructions transmitted to the driving mechanism 506 mayindicate an amount of power to supply to each wheel, a bearing, heading,or where to direct the harvester 100. In this sense, the navigationalsystem 508 may navigate the harvester 100 throughout the field 102 usingthe driving mechanism 506. Such processes may also be carried out by thecomputing system 500 for instructing the driving mechanism 506 as towhere to maneuver the harvester 100. Other navigational instruments mayadditionally, or alternatively, be used for directing the harvester 100within the field 102.

The tracking system 510, meanwhile, may track a location or position ofthe harvester 100 within the field 102. For example, as the harvester100 moves, the tracking system 510 may record or store locations of theharvester 100. These locations may be used to navigate the harvester 100within the field 102, such as along a predetermined route for harvestingthe broccoli plants 104. In some instances, the navigational system 508may utilize the tracking system 510, or data generated by the trackingsystem 510 (e.g., the locations, GPS coordinates, etc.) for navigatingthe harvester 100 within the field 102. For example, based on trackingthe harvester 100, the navigational system 508 may steer to direct theharvester 100 to certain positions or points within the field 102 usingthe driving mechanism 506.

As shown, the computer-readable media 504 may store or otherwise haveaccess to route(s) 512 and field properties 514. In some instances, theroute(s) 512 may represent a route or path the harvester 100 is totravel along within the field 102. The route(s) 512 may, in someinstances, correspond to or be associated with the rows of the broccoliplants 104. The harvester 100 is configured to travel along the route(s)512, and as the harvester 100 travel(s) along the route(s) 512, theharvester 100 may function to harvest the broccoli plants 104. In someinstances, the harvester 100 may travel along the route(s) 512 withoutaid from the operator (e.g., via the driving mechanism 506, thenavigational system 508, and/or the tracking system 510). In someinstances, the route(s) 512 may be predetermined routes based oninformation known about the field 102 and/or the location of the rows ofthe broccoli plants 104 within the field 102. In other instances, theroute(s) 512 may be determined through onboard sensor(s) of theharvester 100 for steering or otherwise directing the harvester 100. Insome instances, the route(s) 512 may be determined such that theharvester 100 may harvest a certain number of rows simultaneously.

In some instances, the route(s) 512 may be determined using the fieldproperties 514. The field properties 514, in some instances, mayrepresent or correspond to a location of the field 102 (amongst otherfields), an amount of rows in the field 102, an amount of plantedbroccoli plants 104 within the field 102, a size of the field 102, anarea of the field 102, dimensions of the field 102 (e.g., shape), and soforth. Knowing the field properties 514, or properties of the field 102,may be used to determine the route(s) 512 along which the harvester 100is to travel to harvest the broccoli plants 104.

The harvester 100 includes imaging system(s) 516 (or devices). Theimaging system(s) 516, as introduced above, images or captures image(s)of the edible crowns 124 or the broccoli plants 104. In some instances,the harvester 100 may include a corresponding number of imagingsystem(s) 516 as the number of rows the harvester 100 is configured toharvest. For example, if the harvester 100 is designed or configured toharvest six rows of broccoli plants 104 simultaneously, the harvester100 may include six imaging systems 516. Additionally, or alternatively,the imaging system(s) 516 may image edible crowns across multiple rowsof broccoli plants 104.

The imaging system(s) 516 may include camera(s) 518 and/or infrared (IR)sensor(s) 520. In some instances, the camera(s) 518 may includered-green-blue (RGB) cameras for capturing colored images of the ediblecrowns. Additionally, or alternatively, the camera(s) 518 may be highdynamic range (HDR) cameras, one or more of light-sensitive cameras,range sensors, or other types of imagers. In some instances, imagescaptured by the camera(s) 518 may be utilized for determining color,size, shape, and/or other characteristic(s) of the edible crowns thatare useful in determining whether the edible crowns are ready forharvesting. The IR sensor(s) 520 may capture depth information, which insome instances, may be used to additionally, or alternatively, determinewhether the edible crowns are ready for harvesting. In some instances,the depth information, depth images, or a depth map generated by the IRsensor(s) 520 may be utilized to determine color, size, shape, and/orother characteristics of the edible crowns that are useful indetermining whether the edible crowns are ready for harvesting. In someinstances, this depth information may be utilized in combination withthe image(s) captured by the camera(s) 518. As such, the computingsystem 500 may use the camera(s) 518 and/or the IR sensor(s) 520 for usein determining whether the edible crowns are ready for harvesting.

In some instances, the imaging system(s) 516 may include multiplecamera(s) and/or multiple IR sensors for imaging the edible crowns (or asingle edible crown) from multiple angles, orientations, and/orpositions. These image(s) may then be analyzed to determine whether theedible crowns are ready for harvesting. In some instances, however, theimaging system(s) 516 may capture a single image of the edible crowns.Regardless of the number of image(s), the number of camera(s), and/orthe number of IR sensor(s) 520 utilized to image the edible crowns, theimage(s) are then processed by components of the computing system 500for determining whether the edible crowns are ready for harvesting. Insome instances, the imaging system(s) 516 may capture videos, and thecomputing system 500 may analyze frames of the videos to determinewhether the edible crowns are ready for harvesting.

The computer-readable media 504 is shown storing image data 522, whichmay correspond to the image(s) captured by the camera(s) 518 and/or theIR sensor(s) 520. Once the imaging system(s) 516 image the ediblecrowns, this information (i.e., the image(s) and/or the depthinformation) may be stored in the computer-readable media 504 for use indetermining whether the edible crowns are ready for harvesting. Thecomputing system 500 may therein analyze the image data 522 of aparticular edible crown to determine whether the particular edible crownis ready for harvesting. As such, the image data 522 may be stored inassociation with particular edible crowns such that the computing system500 may track and record the edible crowns along the route(s) 512, aswell as which image data 522 corresponds to which edible crown along theroute(s) 512 (e.g., to be able to distinguish image data of one ediblecrown from another).

In some instances, the computing system 500 may utilize one or moremachine-learning (ML) model(s) 524 for determining whether to harvestthe edible crowns 124. The ML model(s) 524 may analyze the image data522, or other information, for use in determining whether the ediblecrowns are ready for harvesting. In some instances, the computing system500 may include a scoring component 526 that determines or generatesscores 528 for the edible crowns that are imaged by the harvester 100.In some instances, individual scores 528 may be determined by accessingdata associated with an individual edible crown, such as the image data522, providing the data as input to the ML model(s) 524, and generating,as output from the ML model(s) 524, the score 528 that is associatedwith the individual edible crown.

The score 528 may relate to a probability or likelihood that an ediblecrown is ready for harvesting or not ready for harvesting. In otherwords, the scores 528 determined by the scoring component 526 (e.g.,output by the ML model(s) 524) may be machine-learned scores. Machinelearning generally involves processing a set of examples (called“training data”) in order to train a machine learning model(s). Amachine learning model(s), once trained, is a learned mechanism that canreceive new data as input and estimate or predict a result as output.For example, a trained machine learning model may comprise a classifierthat is tasked with classifying unknown input (e.g., an unknown image)as one of multiple class labels (e.g., labeling the image as a cat or adog). In some cases, a trained machine learning model is configured toimplement a multi-label classification task (e.g., labeling images as“cat,” “dog,” “duck,” “penguin,” and so on). Additionally, oralternatively, a trained machine learning model may be trained to infera probability, or a set of probabilities, for a classification taskbased on unknown data received as input.

In the context of the present disclosure, the unknown input may be theimage data 522 that is associated with a particular edible crown, andthe ML model(s) 524 may be tasked with outputting the score 528 thatindicates, or otherwise relates to, a probability of the edible crownbeing ready for harvesting (or not ready for harvesting). For instance,the score 528 may relate to a probability of an edible crown being readyfor harvesting or not ready for harvesting. The score 528 that is outputby the ML model(s) 524 may relate to either of these probabilities inorder to guide the harvesting processes. If the score 528 that is outputby the ML model(s) 524 relates to a likelihood that the edible crown isready for harvesting, this may indicate that the edible crown is readyfor harvesting.

The training data that is used to train ML model(s) 524 may includevarious types of data. In general, training data for machine learningmay include two components, features and labels. However, in someinstances, the training data used to train the ML model(s) 524 may beunlabeled. Accordingly, the ML model(s) 524 may be trainable using anysuitable learning technique, such as supervised learning, unsupervisedlearning, semi-supervised learning, reinforcement learning, and so on.The features included in the training data can be represented by a setof features, such as in the form of an n-dimensional feature vector ofquantifiable information about an attribute of the training data. Thefollowing is a list of example features that can be included in thetraining data for training the ML model(s) 524 described herein.However, it is to be appreciated that the following list of features isnon-exhaustive, and features used in training may include additionalfeatures not described herein, and, in some cases, some, but not all, ofthe features listed herein. Example features included in the trainingdata may include, without limitation, a width of the edible crown, awidth of the broccoli plant, a length of the edible crown, a length ofthe broccoli plant, a height of the edible crown, a height of thebroccoli plant, a largest-cross-sectional dimension of the edible crown,a largest cross-sectional dimension of the broccoli plant, a color ofthe edible crown (including hue, shade, tint, etc.), a shape of theedible crown, an amount of leaves on the broccoli plant, a row orparticular area within the field 102 at which the edible crown islocated, an amount of buds of the edible crown, a density of the ediblecrown, a volume of the edible crown, a strain or species of the broccoliplant, a number of previously harvested edible crowns at the specific orregional location as the edible crown, a frequency of other ediblecrowns harvested around the edible crown (in the same row or additionalrows), and so forth. In some instances, the features included within thetraining data may be associated with harvested edible crowns (or plants)and/or unharvested edible crowns (or plants).

In some instances, as part of the training process, weights may beapplied to a set of features included in the training data, as derivedfrom the historical data. In some instances, the weights that are setduring the training process may apply to parameters that are internal tothe ML model(s) 524 (e.g., weights for neurons in a hidden-layer of aneural network). These internal parameters of the ML model(s) 524 may ormay not map one-to-one with individual input features of the set offeatures. The weights may indicate the influence that any given feature,parameter, or characteristic has on the score 528 that is output by thescoring component 526 using the ML model(s) 524.

The ML model(s) 524 may represent a single model or an ensemble ofbase-level machine learning models, and may be implemented as any typeof machine learning model. For example, suitable machine learning modelsfor use with the techniques and systems described herein include,without limitation, neural networks, tree-based models, support vectormachines (SVMs), kernel methods, random forests, splines (e.g.,multivariate adaptive regression splines), hidden Markov model (HMMs),Kalman filters (or enhanced Kalman filters), Bayesian networks (orBayesian belief networks), expectation maximization, genetic algorithms,linear regression algorithms, nonlinear regression algorithms, logisticregression-based classification models, or an ensemble thereof. An“ensemble” can comprise a collection of machine learning models whoseoutputs (predictions) are combined, such as by using weighted averagingor voting. The individual machine learning models of an ensemble candiffer in their expertise, and the ensemble can operate as a committeeof individual machine learning models that is collectively “smarter”than any individual machine learning model of the ensemble.

The ML model(s) 524 may learn to identify complex relationships betweencharacteristic(s) of the edible crowns. For example, the ML model(s) 524may learn to associate certain characteristics of the edible crown withone another to indicate whether the edible crown is ready forharvesting. The ML model(s) 524 herein allow for generating the scores528 that more accurately predict whether edible crowns are ready forharvesting, leading to increased yields and fewer instances ofharvesting edible crowns prior to maturity. In some instances, the MLmodel(s) 524 may learn to predict which edible crowns are likely readyfor harvesting, and which edible crowns are unlikely ready forharvesting by attributing corresponding scores 528 to the individualedible crowns. In this manner, edible crowns with low scores (e.g.,below threshold) may not be ready for harvesting, while edible crownswith high scores (e.g., above threshold) may be ready for harvesting.Although the use of a threshold is described as one example way ofproviding labeling (i.e., ready for harvesting or not ready forharvesting), other techniques are contemplated, such as clusteringalgorithms, or other statistical approaches that use the trust scoresfor use in determining whether edible crowns are ready for harvesting.

The ML model(s) 524 is/are retrainable with new data in order to adaptthe ML model(s) 524 to understand harvestable edible crowns, as thecharacteristic(s) of the edible crowns 124 change, or new correlationsbecome available. In some instances, the ML model(s) 524 may beretrained using image data from harvested edible crowns and/orunharvested edible crowns. That is, the ML model(s) may be trained fromcharacteristic(s) of edible crowns that were ready for harvesting and/oredible crowns that were not ready for harvesting, so as to be able todetermine which edible crowns are ready for harvesting and/or whichedible crowns are not ready for harvesting.

In some instances, the harvester 100 may communicatively couple toremote computing resource(s) 530. In some instances, the remotecomputing resource(s) 530 may train the ML model(s) 524 and may thentransmit the ML model(s) 524 to the harvester 100. In some instances,the remote computing resource(s) 530 may train the ML model(s) 524and/or may store data utilized to train the ML model(s) 524. Forexample, over time, one can appreciate that a large collection ofhistorical data tied to harvestable edible crowns may be available tothe remote computing resource(s) 530. The remote computing resource(s)530 may train the ML model(s) 524 using a portion of the historical dataas training data. For instance, a portion of the historical data may belabeled to indicate characteristic(s) of edible crowns that are readyfor harvesting, or which were harvested (and/or not harvested) in thepast. A ML model(s) trained on this data is able to predict harvestableedible crowns by outputting machine-learned scores (e.g., the scores528) that represent a confidence and/or trust that the edible crownsimaged by the imaging system(s) 516 are ready for harvesting. As such,these machine-learned scores are usable by the computing system 500 fordetermining whether the imaged edible crown is ready for harvesting.

After the ML model(s) 524 have been trained, in such instances, theremote computing resource(s) 530 may transmit the ML model(s) 524 to theharvester 100 for use in determining harvestable edible crowns. This mayallow the harvester 100 to determine whether the edible crowns are readyfor harvesting in real-time. However, in some instances, the remotecomputing resource(s) 530 may determine the harvestable edible crowns,using information received from the harvester 100 (e.g., the image data522), and then transmit indications back to the harvester 100 as towhich edible crowns to harvest.

The harvester 100 may communicate with the remote computing resource(s)530 over one or more network(s) 532 and using one or more networkinterface(s) 534. The network(s) 532 may represent and/or include,without limitation, the Internet, other types of data and/or voicenetworks, a wireless infrastructure (e.g., radio frequencies (RF),cellular, satellite, etc.), and/or other connection technologies. Theharvester 100 may, in some instances be part of a network-accessiblecomputing platform that is maintained and accessible via the network(s)532. Network-accessible computing platforms such as this may be referredto using terms such as “on-demand computing”, “software as a service(SaaS)”, “platform computing”, “network-accessible platform”, “cloudservices”, “data centers”, and so forth.

As shown, the computer-readable media 504 may further store or haveaccess to characteristic(s) 558. These characteristic(s) 558 may, insome instances, be determined via the ML model(s) 524 and representcharacteristics of the edible crowns that are imaged by the imagingsystem(s) 516. For example, upon analyzing the image data 522 (e.g., viathe ML model(s) 524), the characteristic(s) 558 of the broccoli plants104 (or of the edible crowns 124) may be determined. By way of example,and as shown, these characteristic(s) 558 may include a color of theedible crown 124, a shape of the edible crown 124, and/or a size of theedible crown 124. However, it is to be understood that the edible crowns124 and/or the broccoli plants 104 may include other characteristicsthat are used for determining whether the edible crowns 124 are readyfor harvesting (or not ready for harvesting). Thus, after analyzing theimage data 522, the characteristic(s) 558 may be determined, which inturn, may be used for determining whether to harvest the edible crowns124. These characteristic(s) 558 may also be used to train and/or retainthe ML model(s) 524.

In some instances, certain characteristics may be indicative of whetherthe edible crown 124 is ready for harvesting, such as color, shape,and/or size. After analyzing the image data 522 to determine thesecharacteristic(s) 558, the characteristic(s) 558 may be compared toreference characteristics, such as a reference size, a reference color,and/or a reference shape. These references may be determined fromtraining the ML model(s) 524 based on the historical data, and may bereferences for comparison to the particular edible crown 124 beingimaged (e.g., the characteristic(s) 558). The ML model(s) 524 mayperform the comparison. The references may be indicative of ediblecrowns 124 that are ready for harvesting and by comparing thecharacteristic(s) 558 to the reference characteristics, the computingsystem 500 may determine whether the edible crowns 124 are ready forharvesting.

In some instances, the scoring component 526 (using the ML model(s) 524)may determine the characteristic(s) 558 and/or analyze thecharacteristic(s) 558 for use in generating the score(s) 528 and/ordetermining whether the edible crowns 124 are ready for harvesting. Forexample, the scoring component 526 may generate the score(s) 528 andcompare the score(s) 528 to a threshold or predetermined level todetermine whether the score 528 satisfies the threshold. If so, theedible crown 124 may be deemed ready for harvesting.

Upon determining that the edible crowns 124 are ready for harvesting,the robotic arms 126 may navigate to and position above the ediblecrowns 124 selected or otherwise flagged for harvesting. The harvester100 may include a corresponding number of the robotic arms 126 as anumber of rows of the broccoli plants 104 that the harvester 100 isconfigured to harvest. In other instances, the harvester 100 may includerobotic arms 126 that harvest edible crowns 124 across multiple rows.The robotic arms 126 may be controlled, or instructed, by the computingsystem 500 or components thereof (e.g., the processor(s) 502). Forexample, after determining that the edible crown 124 is ready forharvesting, the processor(s) 502 may instruct the robotic arm 126 toharvest the edible crown 124. The harvester 100, or the computing system500, may be configured to instruct, simultaneously, the robotic arms 126to harvesting corresponding edible crowns 124 across rows of broccoliplants 104.

As part of instructing the robotic arms 126, the computing system 500may also generate data and/or determine a location of the edible crowns124 relative to the robotic arms 126 (e.g., a position of the roboticarms 126). In some instances, this location may represent coordinatepositions in coordinate space (X, Y, Z) for which the robotic arm 126 isto navigate to in order to harvest the edible crown 124. In someinstances, the computing system 500 may determine the location of theedible crowns 124 to be harvested based on analyzing the image data 522.For example, as part of analyzing the image data 522, and knowing alocation of the imaging system(s) 516 and the robotic arms 126 on theharvester 100, the computing system 500 may determine a location of theedible crown 124 within the field 102.

In some instances, the location of the edible crown 124 may be a centralposition (or point) of the edible crown 124, or a central position ofthe broccoli plant 104. In some instances, the central position of theedible crown 124 may be centered over the edible crown 124 (e.g., X andY positions). The central position may also define a midpoint between atop of the edible crown 124 and a point at which the stalk 122 of thebroccoli plant 104 is to be cut (e.g., just below a base of the ediblecrown 124). Interposed between these two points, may be an additionalcoordinate of the edible crown 124, or the central point (e.g.,Z-position). This point, in the vertical direction, allows the roboticarm 126 to descend unto or onto the edible crown 124 by a certaindistance for grasping the edible crown 124. For example, as discussedabove, the robotic arms 126 include end effectors 536 that function togrip or grasp the edible crowns 124 that are ready for harvesting.

In some instances, the robotic arms 126 may include or be coupled to apositioning system 538 for positioning the end effector 536 relative tothe edible crown 124. In some instances, the positioning system 538 mayrepresent track(s), arms, linkages, members, or rail(s) upon which therobotic arms 126 may move to position the end effector 536. For example,the robotic arm 126 may slide along the tracks and/or rails forpositioning the end effector 536. The positioning system 538 maytranslate in one or more directions for maneuvering the end effector 536relative to the edible crown 124. Additionally, or alternatively, therobotic arms 126 and/or the positioning system 538 may includeactuators, turntables, telescoping assemblies, lifts, and so forth fornavigating the end effector 536 to the edible crown 124. In suchinstances, the computing system 500 may control actuators of the roboticarms 126 and/or of the positioning system 538 for moving the endeffector 536.

The robotic arms 126 may also include a cutting mechanism 540 forharvesting the edible crown 124, or for separating the edible crown 124from the rest of the broccoli plant 104 (e.g., the stalk 122). Thecutting mechanism 540, in some instances, may be disposed on a portionof the end effector 536, and may include one or more stationary bladesand/or one or more actuatable blades actuatable by a motor of actuatorof the cutting mechanism 540 (and/or the robotic arm 126 or the endeffector 536).

After the edible crown 124 is harvested, that is, separated from thestalk 122, the robotic arm 126 may transfer the edible crown 124 to oneor more collection sites. In some instances, the positioning system 538may function to maneuver the robotic arm 126 to the collection sites.

In some instances, after being harvested, the robotic arm 126 maytransfer the edible crown 124 to a flipper 542, which may transfer theedible crown 124 to other portions of the harvester 100 for furtherprocessing. For example, the flipper 542 may include a basket, cradle,or holder for receiving the edible crown 124. The flipper 542 may thenactuate to “flip” the edible crown 124 onto the conveyor belt(s) 130.Upon being transferred to the conveyor belt(s) 130, the edible crown 124may be carried to the platform 128 of the harvester 100 for furtherprocessing.

In some instances, the harvester 100 may include an encoder 544 fortracking or determining a location of the harvester 100 within the field102, or along the route 512. In some instances, the encoder 544 may bepositioned adjected to a hub of one or more of the wheels 108 and maytrack, or record, a rotational movement of the wheel 108. Thisrotational movement may be stored as encoder data 546 within thecomputer-readable media 504. For example, knowing the rotationalmovement or distance traveled by the wheel 108, may assist in knowingthe location of the harvester 100 within the field 102. This locationmay be used for instructing the navigational system 508 to travel to aparticular point within the field 102, and/or may be used fordetermining the position coordinates of the edible crowns 124 forharvesting. For example, knowing the location of the edible crown 124relative to the robotic arm 126 (or the end effector 536) may notaccount for a position of the harvester 100 within the field 102. Thatis, in some instances, the position coordinates of the edible crown 124to be harvested may be determined using both the image data 522 (e.g.,to determine a position of the edible crown 124), as well as locationdata 548 of the harvester 100 (e.g., to determine a position of theharvester 100). In some instances, this location data 548 may bedetermined via the encoder 544 (from the encoder data 546) and/or thenavigational system 508 and/or tracking system 510 (e.g., GPS,triangulation, etc.).

The encoder 544 may also be used for tracking or determining a locationof the edible crowns 124 within the field 102. For example, the encoder544 may track rotational movement of at least one wheel 108 of theharvester 100 to determine distance traveled by the harvester 100 in aforward direction of travel, relative to a reference point. In someinstances, the encoder data 546 generated by the encoder 544 may be usedto determine position coordinates of the edible crowns 124 that aredesignated for harvesting. Additionally, or alternatively, thenavigational system 508 may include a GPS component and the position ofthe harvester 100 within the field 102 may be determined using the GPScomponent. In such instances, the location (the location data 548) maybe determined via the GPS component.

In some instances, the harvester 100 may be configured to performdifferent cut type(s) 550 based on characteristic(s) of the ediblecrowns 124 and/or the broccoli plants 104. These characteristic(s)(e.g., the characteristic(s) 558) may indicate how far down the stalkthe cutting mechanism 540 is to cut and/or whether the end effector 536(or other portions of the robotic arms 126) are to trim/strip leavesfrom the stalk 122. For example, larger sized broccoli plants 104 mayinclude leaves that extend upwards and around/into the edible crown 124.These leaves are often undesirable when harvesting as they may cause anincrease in processing and/or cleaning times. In some instances, basedon analyzing the image data 522, the computing system 500 may determinea size of the edible crown 124 (or the broccoli plant 104) fordetermining which type of cut to perform. For example, if the ediblecrown 124 is ready for harvesting, but a greatest cross-sectionaldimension is less than a threshold (e.g., 4.75 inches), the computingsystem 500 may determine to perform a first type of cut, among the cuttype(s) 550. The computing system 500 may then instruct the robotic arm126, or the cutting mechanism 540 to perform the first type of cut.Alternatively, if the greatest cross-sectional dimension is greater thanthe threshold, the computing system 500 may determine to perform asecond type of cut among the cut type(s) 550. The computing system 500may then instruct the robotic arm 126, or the cutting mechanism 540 toperform the second type of cut. In some instances, the second type ofcut may involve stripping leaves around the edible crown 124 using theend effector 536. Alternatively, certain consumers may desire longerstalks 122 attached to the edible crown 124. In such instances, thecutting mechanism 540 may be configured to cut varying lengths of stalk122 below the edible crown 124.

The harvester 100 may also include lighting element(s) 552. In someinstances, the lighting element(s) 552 may illuminate the edible crowns124 underneath the hood 120 for obtaining quality image(s) of the ediblecrowns 124. The lighting element(s) 552 may also illuminate the ediblecrowns 124 in instances where the harvester 100 operates at night or lowlighting conditions (e.g., overcast). The lighting element(s) 552, insome instances, may comprise white light LEDs and/or colored LEDS. Thelighting element(s) 552 may also include organic light emitting diodes(OLEDs), and/or other lights that adequately illuminate the ediblecrowns 124.

The computer-readable media 504 may further store harvest data 554,which may include harvested edible crowns and unharvested edible crowns.The harvested edible crowns may correspond to those edible crowns 124that were harvested, while the unharvested edible crowns may correspondto those edible crowns 124 that were not harvested and which remainplanted in the field 102 (e.g., unharvested edible crowns 206). In someinstances, the harvest data 554 may indicate a location of the harvestededible crowns as well as a location of the unharvested edible crowns.Such locations may be used for analyzing field properties and/orcharacteristics, for use in future instances when harvesting. Thisinformation may also be used to train the ML model(s) 524. Additionally,the information may be used to determine which portions of the field 102have a high yield, which portions have a low yield, which portions ofthe field 102 were harvested, and/or which portions of the field 102were not harvested. These trends may be analyzing for adjustingharvesting schedules and/or other harvesting processes (e.g., watering,fertilizing, etc.). Additionally, noted above, the harvest data 554 mayidentify characteristics of the harvested edible crowns and unharvestededible crowns. This information may be used to retain the ML model(s)524 and/or may be used to determine when the unharvested edible crownwill be ready for harvesting. Therein, at future instances, theharvester 100 or personnel may selectively broccoli harvest the ediblecrowns at they become ready.

The harvester 100 may also include a de-leafing component 556 thatremoves leaves from the broccoli plant 104. In some instances, thede-leafing component may be located in front of the imaging system(s)516, relative to the direction of travel of the harvester 100, to removethe leaves and isolate the edible crown 124 for obtaining clearimage(s). In some instances, the de-leafing component 556 may be aseparate component, machine, or device than the harvester 100. Theimaging system(s) 516 may image the edible crown 124 for use by thecomputing system 500 to determine whether the edible crown 124 is readyfor harvesting. In some instances, the de-leafing component 556 mayinclude rotating blade(s) that may resemble rotary knives or swingingflail knives. In some instances, the rotating blades are spaced apart bya distance such that the rotating blades pass along a row of thebroccoli plants 104 to cut away the outer leaves of the broccoli plant104 and leave the broccoli plants 104 unharmed and revealing the ediblecrown 124.

As used herein, the computer-readable media 504 may be implemented ascomputer-readable storage media (“CRSM”), which may be any availablephysical media accessible by the processor(s) 502 to executeinstructions stored on memory. The memory (or non-transitorycomputer-readable media) may include volatile and nonvolatile memory,removable and non-removable media implemented in any method ortechnology for storage of information, such as computer-readableinstructions, data structures, program modules, or other data. Suchmemory includes, but is not limited to, random access memory (“RAM”),read-only memory (“ROM”), erasable programmable read-only memory(“EEPROM”), flash memory or other memory technology, CD-ROM, digitalversatile disks (DVD) or other optical storage, magnetic cassettes,magnetic tape, magnetic disk storage or other magnetic storage devices,RAID storage systems, or any other medium which can be used to store thedesired information and which can be accessed by a computing device andthe processor(s) 502.

FIGS. 6A and 6B illustrate the harvester 100 traveling within a field600 to harvest edible crowns. As shown, the field 600 may include twelverows of broccoli plants, and the harvester 100 may be configured toimage and harvest edible crowns across the twelve rows of broccoliplants. The wheels 108 of the harvester 100 are shown spanning acrossthe twelve rows of broccoli plants.

The harvester 100 travels along a route 602 within the field 600. Theharvester 100 may, in some instances, include a first hood 604 forimaging edible crowns along the route, and as the harvester 100 travelsin a direction of travel (Y-direction, and as shown by the arrow). Thefirst hood 604 may include similar components as the hood 120 (e.g., theimaging system(s) 516, etc.). Additionally, in some instances, theharvester 100 may include a second hood 606, which may include similarcomponents as the additional hood 200.

As the harvester 100 travels along the route 602, the harvester 100images the edible crowns and determines whether to harvest individualedible crowns. For example, as shown in FIG. 6B, the harvester 100includes a first imaging system 608(1) for imaging edible crowns withina first row 610(1) and a second imaging system 608(2) for imaging ediblecrowns within a second row 610(2). Once the edible crowns pass under (orcome within a field of view of the first imaging system 608(1) and thesecond imaging system 608(2)) the first hood 604, the first imagingsystem 608(1) and the second imaging system 608(2) may image the ediblecrowns of the first row 610(1) and the second row 610(2), respectively.

The image(s) captured by the first imaging system 608(1) and the secondimaging system 608(2) may be analyzed by the computing system 500 fordetermining whether the edible crowns are ready for harvesting. If so,robotic arms and end effectors may harvest the edible crowns. Forexample, as shown in FIG. 6B, after being imaged by the first imagingsystem 608(1) and the second imaging system 608(2), the harvester 100may include a first robotic arm 612(1) and a second robotic arm 612(2)for harvesting the edible crowns. The first robotic arm 612(1) and/orthe second robotic arm 612(2) may be similar to and/or include similarcomponents as the robotic arm 126. In some instances, the first roboticarm 612(1) and the second robotic arm 612(2) may be disposed between thefirst hood 604 and the second hood 606. Although the first robotic arm612(1) and the second robotic arm 612(2) are shown including endeffectors (e.g., the end effectors 536) having two fingers for graspingthe edible crown, the first robotic arm 612(1) and the second roboticarm 612(2) (or the end effectors) may include more than or less thanfour fingers. The fingers are sized and configured to descend over a topof the edible crown and converge around the edible crown for harvesting.

In some instances, the first robotic arm 612(1) may be associated withthe first row 610(1) and for harvesting edible crowns within the firstrow 610(1), after being imaged by the first imaging system 608(1). Thesecond robotic arm 612(2) may be associated with the second row 610(2)and for harvesting edible crowns within the second row 610(2), afterbeing imaged by the second imaging system 608(2). The first robotic arm612(1) and the second robotic arm 612(2) may include respective endeffectors, cutting mechanisms, actuators, and so forth, for harvestingthe edible crowns.

As shown in FIG. 6B, some of the edible crowns within the field 600 maynot be ready for harvesting. Consequentially, after being imaged (e.g.,by the imaging systems of the harvester 100) these edible crowns may notbe harvested by the harvester 100. For example, FIG. 6B illustrates thaton a trailing side of the harvester 100, broccoli plants 614 may remainplanted for further growing. These broccoli plants 614 may then beharvested at later instances when ripe. Accordingly, FIGS. 6A and 6Billustrate that as the harvester 100 travels throughout the field 600,the computing system 500 may function to determine which edible crownsto harvest and which edible crowns are to remain planted.

FIG. 7 illustrates the bottom 400 of the hood 120, showing an imagingsystem 700 for capturing images of one or more edible crowns within oneor more rows of the broccoli plants 104. The imaging system 700 mayrepresent, be similar to, and/or include similar components as theimaging system 516.

In some instances, and as shown in FIG. 7, two rows of broccoli plantsmay be planted in close proximity to one another, spaced apart fromother row of broccoli plants (X-direction). In some instances, theimaging system 700 may include a first camera/IR sensor 702(1) forcapturing images of a first row 704(1) of the broccoli plants and asecond camera/IR sensor 702(2) for capturing images of a second row ofthe broccoli plants 104. The first camera/IR sensor 702(1) may imagefirst edible crowns (or the broccoli plants) within the first row704(1), while the second camera/IR sensor 702(2) may image second ediblecrowns within the second row 704(2). In some instances, the firstcamera/IR sensor 702(1) may be oriented (e.g., angled, tilted, etc.) forcapturing images of the broccoli plants within the first row 704(1) andthe second camera/IR sensor 702(2) may be oriented (e.g., angled,tilted, etc.) for capturing images of the broccoli plants within thesecond row 704(2). In some instances, the first camera/IR sensor 702(1)and/or the first camera/IR sensor 702(2) may include an RGB cameraand/or an IR sensor.

The imaging system 700 may be located in front of, relative to thedirection of travel (Y-direction), the robotic arms 126 that harvest theedible crowns. The hood 120 may include a leading or a front edge 706and a trailing or a back edge 708. Additionally, discussed above inrelation to FIG. 4, the hood 120 may include the first end 402 and thesecond end 404 (not shown), located opposite the first end 402. In someinstances, the imaging system 700 may be centered between the front edge706 and the back edge 708.

The first camera/IR sensor 702(1) and/or the second camera/IR sensor702(2) may capture images of one or more edible crowns, within the firstrow 704(1) and the second row 704(2), respectively, for use determiningwhether the one or more edible crowns are ready for harvesting.Additionally, or alternatively, in some instances, the imaging system700 may take multiple images using the first camera/IR sensor 702(1),the second camera/IR sensor 702(2), and/or one or more additionalcamera(s) and may combine the images (e.g., stitching) for analyzing theedible crowns. In such instances, image(s) from multiple camera(s) maybe used for determining whether to harvest a particular edible crown.Additionally, or alternatively, in some instances, the first camera/IRsensor 702(1) and the second camera/IR sensor 702(2) may captureimage(s) of a single edible crown within the first row 704(1) and thesecond row 704(2), respectively for determining whether the ediblecrowns 124 are ready for harvesting. For example, as the broccoli plantspass underneath the hood 120 and come within a field of view of theimaging system 700, image(s) of the broccoli plants 104 may be captured.

In some instances, the imaging system 700 may image the broccoli plantsat a predetermined offset from the front edge 706. For example, theimaging system 700 may wait to image the broccoli plants 104 until theedible crowns 124 are directly underneath (X and Y-directions) the firstcamera/IR sensor 702(1) and the second camera/IR sensor 702(2),respectively, or are aligned with the first camera/IR sensor 702(1) andthe second camera/IR sensor 702(2) (Y-direction). Alternatively, in someinstances, the first camera/IR sensor 702(1) and the second camera/IRsensor 702(2) may image the broccoli plants at a predetermined offsetfrom the first camera/IR sensor 702(1) and the second camera/IR sensor702(2), respectively.

In some instances, the field of view of the imaging system 700 may beadjusted, and accordingly, a point or position at which the firstcamera/IR sensor 702(1) and/or the second camera/IR sensor 702(2) imagesthe broccoli plants may be adjusted. The adjustment may account forlighting conditions, environmental conditions, and/or characteristics ofthe broccoli plants within the rows. For example, waiting until thebroccoli plants 104 are completely underneath the hood 120 may reduce animpact on external weather (e.g., sun, rain, wind) affecting theimage(s) (or quality of the image(s)) captured. Furthermore, spacing inbetween the broccoli plants 104 within the same row may be different,and the first camera/IR sensor 702(1) and/or the second camera/IR sensor702(2) may have to be adjusted to accommodate image capturing of thebroccoli plants 104.

To account for the adjustments of image capturing, the imaging system700 may be coupled to actuators or motors that adjust the field ofviews. In some instances, the imaging system 700 may adjust alongmultiple axes (X, Y, and Z) and/or in multiple degrees of freedom (e.g.,pan, tilt, yaw, etc.). In some instances, each of the first camera/IRsensor 702(1) and the second camera/IR sensor 702(2) may beindependently actuatable for controlling or adjusting the field of viewof each of the first camera/IR sensor 702(1) and the second camera/IRsensor 702(2), respectively. For example, mounts, brackets, gears,slides, tracks, motors, wheels, pulleys, pneumatics, hydrauliccylinders, cables, screw drives, turntables, or other actuators mayposition, move, or orient the imaging system 700.

In some instances, the imaging system 700 may include lightingelement(s) (e.g., the lighting element(s) 552) for illuminating theedible crowns or a portion of the edible crowns imaged by the imagingsystem 700. In some instances, the lighting element(s) may comprisewhite light LEDs or may include colored LEDs. Additionally, portions ofthe hood 120 above the edible crowns (e.g., on the bottom 400) mayinclude blowers for removing debris, moisture, or other foliage forvisibly capturing image(s) of the broccoli plants.

As discussed above, after capturing images of the edible crowns withinthe first row 704(1) and the second row 704(2), the computing system 500may analyze the image(s). That is, after being imaged and while theharvester 100 moves, the edible crowns 124 pass through from underneaththe hood 120, past the back edge 708 of the hood 120, and if ready forharvesting, may be harvested by the robotic arms 126.

Although FIG. 7 only illustrates one imaging system 700, the harvester100 (or the hood 120) may include a corresponding number of imagingsystems 516 for imaging broccoli plants. For example, a second imagingsystem may include a third camera/IR sensor for imaging edible crownswithin a third row and/or a fourth camera/IR sensor for imaging ediblecrowns within a fourth row. This second imaging system may be spacedapart from the imaging system 700 in a direction towards the second end404. Additionally, the imaging system(s) may be spaced apart by a knowndistance interposed between rows of broccoli plants. However, notedabove, the imaging system(s) may be configured to move along multipleaxes to reposition and image the edible crowns.

FIGS. 8A and 8B illustrate a diagram for aligning an end effector 536relative to an edible crown 800 ready for harvesting. Initially, afterdetermining that the edible crown 800 is ready for harvesting, thecomputing system 500 (or another component of the harvester 100) maydetermine a location associated with harvesting the edible crown 800.The end effector 536 may then position to the location (e.g., via arobotic arm, positioning system, etc.) to harvest the edible crown 800.

In some instances, the location of the edible crown 800 may bedetermined based at least in part on analyzing the image data 522,encoder data 546, and/or location data 548. For example, the computingsystem 500 may analyze the image data 522 for determining a location ofthe edible crown 800. The image data 522 may correspond to colored imagedata and/or a depth map, for determining a relative distance between theimaging system 516 and the edible crown 800 being imaged. This distancemay be usable for determining a location of the edible crown 800relative to the robotic arm 126, the end effector 536, or other portionsof the harvester 100.

Additionally, or alternatively, the computing system 500 may analyze GPScoordinates of the navigational system 508 for determining a location ofthe harvester 100. This location may, in some instances, be relative tothe end effector 536. In other instances, the location may be associatedwith a location of the edible crown 800. As such, the location of theedible crown 800 may be determined relative to the end effector 536 orrelative to the harvester 100 for instructing or causing the endeffector 536 to maneuver to the location for harvesting the edible crown800.

The end effector 536 is shown positioned at an origin or restingposition. In some instances, the origin may correspond to (0, 0, 0) inthe (X, Y, Z) coordinate space. The end effector 536 may be configuredto move from the origin to a location associated with the edible crown800. For example, discussed above, the end effector 536 may operablycouple to the robotic arm 126 and/or positioning system 538 thatfunctions to move the end effector 536. As shown, the edible crown 800may have a center point (X₁, Y₁, Z₁) in coordinate space. In someinstance, the center point may correspond to a center of mass and/orcenter of volume of the edible crown 800.

After determining the center point, the end effector 536 may effectuateto move to the center point. For example, if the origin is (0, 0, 0) inthe (X, Y, Z) coordinate space, as shown in FIG. 8A, the end effector536 may be moved by a distance X₁ in the X-direction and a distance Y₁in the Y-direction. After moving by these distances in the X-directionand the Y-direction, respectively, the end effector 536 may besubstantially centered above the edible crown 800 (e.g., disposedvertically above the edible crown 800). In some instances, the endeffector 536 may first be moved in the X-direction and then in theY-direction, or may first be moved in the Y-direction and then in theX-direction.

After being disposed vertically above the edible crown 800, the endeffector 536 may descend downward upon the edible crown 800 in theZ-direction, by a distance Z₁. As shown in FIG. 8B the position Z₁ maycorrespond to a center of mass or center point of the edible crown 800.In some instances, the end effector 536 may not actually extend to thepoint Z₁. For example, extending to the point Z₁ may cause the endeffector 536 to hit the edible crown 800, causing damage. Rather, theend effector 536 may include a center point for aligning with the centerpoint of the edible crown 800. Therein, when these center points arealigned, the edible crown 800 may be positioned within the end effector536, or within a center of the end effector 536.

In some instances, the distance Z₁ may correspond to a distance suchthat when the end effector 536 encloses around the edible crown 800, theend effector 536 engages with a stalk of the broccoli plant just belowthe edible crown 800. However, this distance may be varied to cutvarying lengths of the stalk. Once the end effector 536 descends by thedistance Z₁, the cutting mechanism 540 may sever the edible crown 800from a remaining portion of the broccoli plant. Therein, the endeffector 536 may travel to a collection point for transferring theedible crown 800.

The end effector 536 may then travel to a new location associated withharvesting a different edible crown, such as a subsequent edible crown802 within the same row as the edible crown 800. For example, based onanalyzing image data 522 of the subsequent edible crown 802, the encoderdata 546, the location data 548, and/or GPS coordinates, a center point(X₂, Y₂, Y₂) of the subsequent edible crown 802 may be determined. Insome instances, the end effector 536 may return to the origin (0, 0, 0)after transferring the edible crown 800 at the collection point, andbefore traveling to the center point of the subsequent edible crown 802.In other instances, the end effector 536 may be configured to travelstraight from the collection point to the center point of the subsequentedible crown 802. As such, the end effector 536 may be controlled in athree-dimensional coordinate space for harvesting edible crowns on acontinuous basis.

Although the above discussion relates to positioning a single endeffector 536, it is to be understood that the computing system 500 mayrespectively control a plurality of end effectors for harvesting aplurality of edible crowns across a plurality of rows. Furthermore,although the end effector 536 is shown including two fingers forgrasping the edible crown 800, the end effector 536 may include morethan or less than two fingers. As shown, the fingers of the end effector536 may be shaped and/or profiled for cradling the edible crown 800 oncecut. Additionally, the fingers may be oriented differently than shown.

FIG. 9 illustrates a diagram 900 of successive steps for harvesting anedible crown. In some instances, steps illustrated in the diagram 9000may be performed once the edible crown is determined to be harvested.

In some instances, at “1” the first step in harvesting the edible crownmay include determining coordinates (e.g., X, Y, Z) associated with theedible crown. The coordinates, in some instances, may include a (X, Y,Z) center point of the edible crown. For example, after determining thatthe edible crown is ready for harvesting, the coordinates associatedwith harvesting the edible crown may be determined. As discussed above,the coordinates may be determined utilizing the image data 522, thelocation data 548 (e.g., GPS coordinates), and/or the encoder data 546.However, other methods may be used for determining the coordinates forinstructing components of the harvester 100 to harvest the edible crown.

At “2” the second step in harvesting the edible crown may includepositioning the end effector along a first axis (or plane) to align withthe edible crown. For example, as shown, the end effector 536 may movein a first direction (e.g., in the X-direction) to align with the ediblecrown in a first direction, or align the with an X-plane extendingthrough the center point. In some instances, aligning the end effector536 with the edible crown in the first direction may include actuatingthe robotic arm 126 and/or the positioning system 538. For example, thecomputing system 500 may cause the end effector 536 to align with theedible crown in the first direction by instructing or actuating therobotic arm 126 and/or the positioning system 538.

At “3” the third step in harvesting the edible crown may includepositioning the end effector 536 along a second axis (or plane) to alignwith the edible crown. For example, as shown, the end effector 536 maymove in a second direction (e.g., in the Y-direction) to align with theedible crown in a second direction, or align with a Y-plane extendingthrough the center point. In some instances, aligning the end effector536 with the edible crown in the second direction may include actuatingthe robotic arm 126 and/or the positioning system 538. For example, thecomputing system 500 may cause the end effector 536 to align with theedible crown in the second direction by instructing or actuating therobotic arm 126 and/or the positioning system 538. In some instances,the steps “2” and “3” may be performed in reverse order, simultaneously,and/or substantially simultaneously.

In some instances, at “3” the end effector 536 may move in the seconddirection along the first axis (e.g., the X-axis). That is, the endeffector 536 may maintain the alignment with the X-coordinate positionof the center point while the end effector 536 moves in the seconddirection. As such, after moving in the second direction, the endeffector 536 may be centered (or substantially centered) above theedible crown, in the first direction (or along the first axis/plane) andthe second direction (or along the second axis/plane).

At “4” the fourth step in harvesting the edible crown may includedescending the end effector 536 to align the end effector 536 along athird axis (or plane) associated with the edible crown. For example, asshown, the end effector 536 may descend in a third direction (e.g., inthe Z-direction) to align with the edible crown in a third direction, oralign with a Z-plane extending through the center point. In someinstances, aligning the end effector 536 with the edible crown in thethird direction may include actuating the robotic arm 126 and/or thepositioning system 538. For example, the computing system 500 may causethe end effector 536 to align with the edible crown in the thirddirection by instructing or actuating the robotic arm 126 and/or thepositioning system 538.

At “4” the end effector 536 may move in the third direction along thefirst axis (e.g., the X-axis/plane) and the second axis (e.g., theY-axis/plane). That is, the end effector 536 may maintain the alignmentwith the X-coordinate position and the Y-coordinate position of thecenter point while the end effector 536 moves in the third direction. Assuch, after moving in the third direction, the end effector 536 may becentered (or substantially centered) on the edible crown, in the firstdirection, the second direction, and the third direction. Here, the endeffector 536 may be disposed around, or substantially enclose, theedible crown. As shown, the end effector 536 at “4” may be in the openposition and the fingers of the end effector 536 may be sized and spacedapart to allow the end effector 536 (and the fingers) to descend uponthe edible crown.

At “5” the fifth step in harvesting the edible crown may includeenclosing the end effector 536 on the edible crown and severing theedible crown from the stalk. For example, after disposed over the ediblecrown, actuators may actuate components of the end effector 536 toenclose around the edible crown. Thereafter, the cutting mechanism 540may cut the stalk to separate the edible crown.

Although the end effector 536 is shown including two fingers forgrasping the edible crown, the end effector 536 may include more than orless than two fingers, such as four fingers.

FIG. 10 illustrates an example diagram showing a route of the harvester100 across rows of broccoli plants, or within a field 1002 of broccoliplants. As shown, initially, the harvester 100 may be aligned on a firstside 1000 of the field 1002. In some instances, the harvester 100 may beconfigured to harvest multiple rows of broccoli plants at the same time.For example, as shown in FIG. 10, the harvester 100 may be configured tosimultaneously harvest twelve rows of broccoli plants.

The harvester 100 is configured to harvest the broccoli plants by movingthrough the field 1002, from the first side to a second side 1004. Forexample, at “1” the harvester 100 may travel from the first side 1000 tothe second side 1004 (Y-direction). Traveling at “1” may besubstantially in a first direction, such as the Y-direction.

Once on the second side 1004, at “2” the harvester 100 may travel in asecond direction, such as the X-direction. Traveling at “2” may centerthe harvester 100 for harvesting another twelve rows of broccoli plants,as the harvester 100 travels from the second side 1004 to the first side100.

In some instances, after harvesting the first twelve rows at “1” theharvester 100 may pivot the wheels 90 degrees, for example, and travelin the X-direction. In this sense, the harvester 100 may avoid turningaround, performing a “U-turn”, and so forth. Rather, the wheels 108 ofthe harvester 100 may be independently steerable to reduce a turningradius of the harvester 100. As such, after harvesting rows of broccoliplants, the harvester 100 may simply turn the wheels 108 (e.g., 90degrees clockwise), travel to other rows for harvesting, and the turnthe wheels 108 back (e.g., 90 degrees counterclockwise).

Stated alternatively, because the harvester 100 may include multiplehoods (e.g., the hood 120 and the additional hood 200), the harvester100 may not include a designated “front” and “back” or a particulardirection of travel. That is, because the harvester 100 may include twohoods with imaging systems 516, the harvester 100 may not have a singledirection of travel.

For example, at “3” the harvester 100 may travel from the second side1004 to the first side 1000 to harvest an additional twelve rows ofbroccoli. At “3” the harvester may utilize different components forharvesting the broccoli plants. That is, because the harvester 100 ismoving in a different direction of travel at “3” as compared to “1”,different components may be used for imaging and/or harvesting thebroccoli plants. For example, for harvesting the broccoli plants at “1”the harvester 100 may utilize imaging systems 516 on the hood 120 (e.g.,a first hood), and harvesting the broccoli plants at “3” may utilizeimaging systems 516 on the additional hood 200 (e.g., a second hood).

Therein, at “4” the harvester 100 may turn the wheels for aligning withanother twelve rows of broccoli, and at “5,” may harvest the broccoliplant. At “5” the harvester 100 may be traveling in a substantiallysimilar direction of travel as “1” and therefore, may utilize the sameimaging system 516 at “5” used to image the edible crowns at “1”.

Therefore, from “1” to “5” the harvester 100 may harvest the broccoliplants without performing wide turns in between harvesting rows ofbroccoli plants. Instead, the wheels 108 may reposition after harvestingthe rows of broccoli plants to limit an amount of time the harvester 100takes before harvesting addition rows of broccoli plants.

FIGS. 11A and 11B illustrate the de-leafing component 556 for de-leafingbroccoli plants or removing leaves from the broccoli plants. Asdiscussed above, broccoli plants have an edible flower formed at the tipof the broccoli stalk. However, healthy broccoli plants often have anabundance of leaves growing off the stalk that reside beneath, alongsideof, and even above the edible crown. These leaves may obstruct theedible crown, which may impact the ability to accurately determinewhether the edible crown is ready for harvesting.

For example, in some instances, the leaves may block or hinder theimaging system 516 being able to obtain a clear image of the ediblecrown. The subsequently obtained image(s) may fail to clearly indicatewhether the edible crown is ready for harvesting. In turn, the computingsystem 500 may inaccurately determine that the edible crown is ready forharvesting or not ready for harvesting. Therefore, obtaining clearimage(s) of the edible crown is important when analyzing whether theedible crown is ready for harvesting. The de-leafing component 556 maytherefore remove leaves around the edible crown for obtaining clear andunobstructed image(s).

As shown in FIG. 11A, which represents a side view of the harvester 100,broccoli plants 1100 may pass underneath a hood 1102. The hood 1102 maybe similar to and/or include similar components as the hood 120. Asshown by the direction of travel, the broccoli plants 1100 may enter thehood 1102 having leaves disposed on the broccoli stalk and/or around theedible crown of the broccoli plants 1100. As the broccoli plants 1100enter underneath the hood 120 the de-leafing component 556 may removeunwanted leaves. The de-leafing component 556 is shown extending fromthe hood 1102 in a direction towards the broccoli plants 1100 (e.g.,towards the ground).

The de-leafing component 556 may include rotating blades 1104. In someinstances, the rotating blades 1104 may resemble rotary knives orswinging flail knives. In some instances, the rotating blades 1104 maybe spaced apart by a distance such that the rotating blades 1104 passalong a row of the broccoli plants 1100 to cut away the outer leaves ofthe broccoli plant 1100, leaving the broccoli plant 1100 unharmed andrevealing the edible crown.

The rotating blades 1104 may be located on a shaft 1106 of thede-leafing component 556 that may be configured to rotate when powered.As shown in FIG. 11A, in some instances, each de-leafing component 556may include a series of rotating blades 1104 that are disposed apartfrom one another along a length of the shaft 1106. For example, in someinstances, the de-leafing component 556 may include three rotatingblades 1104 that are spaced apart along the length of the shaft 1106(Z-direction). In some instances, some of or all of the blades 1104 maybe rotating blades, while some of or all of the blades may be stationaryand/or fixed on the shaft 1106. In some instances, spacing the rotatingblades 1104 in this manner may serve to remove leaves that extend upwardtowards the edible crown, and/or at different portions along the lengthof the stalk.

In some instances, the de-leafing component 556 may not remove all ofthe leaves from the stalk, but those leaves that extend close to theedible crown, or which are located near the top of the broccoli plant1100. For example, the de-leafing component 556 may not remove leavesfrom the stalk that are located near the base of the broccoli plants1100 (e.g., near the ground). Additionally, or alternatively, thede-leafing component 556 may be configured to remove outer leaves, suchthat when removed, no longer occlude the edible crown.

After the de-leafing component 556 removes the leaves, as shown in FIG.11A, the leaves may be removed from the stalk. Therein, the imagingsystem 516 may image the edible crown as the edible crown pass or comewithin a field of view of the imaging system 516. Therein, as discussedabove, the computing system 500 may determine whether to harvest theedible crown, and if so, the computing system 500 may instruct therobotic arm 126 to harvest the edible crown (via the end effector 536and the cutting mechanism 540). As such, the de-leafing component 556may be located in front of the imaging system 516 for removing theleaves to isolate the edible crown and prior to imaging.

FIG. 11B illustrates a top view of the hood 1102 and illustratescomponents of the hood 1102 in dashed lines to indicate the broccoliplants 1100 disposed therebeneath (Z-direction). In some instances, theharvester 100 may include de-leafing components 556 for individual rowsof the broccoli plants 1100 being harvested. For example, the harvester100 may include one or more de-leafing components 556 for each of therows of broccoli plants 1100 being harvested. In some instances, theharvester 100 may include two de-leafing components for each row ofbroccoli plants 1100 being harvested. In such instances, a firstde-leafing component 556 may be located on a first side of the broccoliplant 1100, or a first side of the row, while a second de-leafingcomponent 556 may be located on a second side of the broccoli plant1100, or a second side of the row. As the harvester 100 moves in thedirection of travel the broccoli plants 1100 pass underneath the hood120 whereby the de-leafing component(s) 556 may remove the leaves. Afterthe leaves are removed, the imaging systems 516 may image the ediblecrowns.

In some instances, the de-leafing components 556 may be stationary(e.g., fixed position) or may be configured to move in one or moredirections to trim or cut the leaves from multiple side(s) of thebroccoli plants 1100. For example, information about the spacing of thebroccoli plants 1100 within the rows (e.g., spacing between individualrows, spacing across rows, etc.) may be used to pre-position thede-leafing components 556 for oncoming broccoli plants 1100. Moreover,information about a height of the broccoli plants 1100 or a thickness ofthe stalk may be used to position the de-leafing components 556. In someinstances, the de-leafing components 556 may not sever the leaves at aposition where the leaves extend from the stalk (e.g., the leaves maynot be cut off flush with the stalk). Rather, in some instances, theleaves may be cut off at positions that are halfway, two-thirds, etc.along the length of leaves. However, the de-leafing components 556 mayeffectuate to remove the portion of the leaves extending upward towardsthe edible crown, or which obstruct the edible crown. Therefore,removing the entire leaf from the stalk may be unnecessary.

In some instances, the de-leafing components 556 may be continuouslypowered given the continuous movement of the harvester 100. Theharvester 100 may also include blowers or other fans for removingremints of the leaves once cut. For example, after the leaves are cut,remints may be located on the edible crown and may impact a quality ofimage(s) obtained. The blowers may remove unwanted leaves (or portionsthereof) from the edible crown or area(s) in proximity to the ediblecrown.

In some instances, the de-leafing components 556 may also, in someinstances, increase harvesting times. For example, as the de-leafingcomponents 556 remove leaves, once harvested, the edible crowns may nothave to be trimmed (either manually or with a separate machine) toremove unwanted leaves from the edible crown.

Although FIGS. 11A and 11B illustrate certain components or embodimentsof the de-leafing components 556, the de-leafing components 556 mayinclude other components and/or may be embodied differently. Forexample, rather than including the rotating blades 1104, the de-leafingcomponents 556 may use stationary blades for removing the leaves. Insome instances, the de-leafing components 556 may only remove leavesfrom one side, or multiple sides of the broccoli plants 1100. Forexample, it may be difficult to trim leaves on a leading or trailingside of the broccoli plants 1100, relative to the direction of travel.However, in some instances, the de-leafing components 556 may be coupledto, or include, actuators that move the de-leafing components 556 aroundthe broccoli plants 1100 (e.g., to a front of the broccoli plants 1100).Additionally, or alternatively, in some instances, the harvester 100 mayinclude more than or less than two de-leafing component 556 per row ofthe broccoli plants 1100, or the de-leafing components 556 may removeleaves across one or more rows. For example, the de-leafing components556 may remove leaves from a first broccoli plant located in a first rowof the broccoli plants 1100 and remove leaves from a second broccoliplant located in a second row of the broccoli plants 1100.

Furthermore, the de-leafing components 556 may also be located elsewhereon, or under, the hood 1102 and/or on other portions of the harvester100. In some instances, the de-leafing components 556 may be an externaldevice, machine, or apparatus, and which may not be coupled to the hood1102 and/or the harvester 100. In some instances, the harvester 100 mayinclude components for imaging the broccoli plants 1100 for firstdetermining whether the broccoli plants 1100 need to be trimmed, orwhether the leaves obscure the edible crown. That is, if the leaves arenot obstructing the edible crown (i.e., a clear image may be obtained),then the leaves may not be removed and/or the de-leafing components 556may not be actuated. If, however, the leaves are obstructing the ediblecrown, the de-leafing components 556 may be actuated to remove theleaves.

The blades of the de-leafing components 556 may also be different thanas shown in FIGS. 11A and 11B (e.g., size, shape, angles, etc.). Forexample, individual de-leafing components 556 may include a first rotary(or stationary) blade arranged substantially horizontally relative tothe shaft 1106, a second rotary blade arranged substantially verticallyrelative to the shaft 1106, and/or a third rotary blade disposed atanother angle relative to the shaft 1106. In some instances, the blades1104 may rotate about the shaft 1106, or may rotate at various anglesrelative to the shaft 1104 (e.g., perpendicular). These arrangements mayserve to cut the leaves from the stalk to accommodate for the differentgrowing characteristics of the leaves.

FIGS. 12A and 12B illustrate a detailed view of a robotic arm 1200 ofthe harvester 100. In some instances, the robotic arm 1200 may besimilar to, represent, and/or include features as described above withregard to the robotic arm 126.

The robotic arm 1200 includes an end effector 1202. The end effector1202 may be similar to, represent, and/or include features as describedabove with regard to the end effector 536. The end effector 1202 isconfigured to transition between an open position. FIG. 12A illustratesthe end effector 1202 in the closed position and FIG. 12B illustratesthe end effector 1202 in the open position.

Generally, and as discussed above with regard to the end effector 536,the end effector 1202 may represent a gripper that grasps edible crownsof broccoli plants ready for harvesting. In the open position, the endeffector 1202 may descend unto or over edible crowns. In the closedposition, the end effector 1202 may grasp onto edible crowns (orportions of the stalk) for retaining edible crowns within the endeffector 1202.

In some instances, the end effector 1202 may include a body 1204 andfingers attached to the body 1204. The body 1204 may include one or morearms or wings that extend outward for receiving or coupling to thefingers 1206. In some instances, the end effector 1202 may include twofingers as shown in FIGS. 12A and 12B, such as a first finger 1206(1)and a second finger 1206(2) (collectively, referred to herein as “thefingers 1206”). However, in some instances, the end effector 1202 mayinclude more than two fingers, such as three fingers, four fingers,and/or any other number of fingers.

The fingers 1206 may be equidistantly spaced apart from one another onthe body 1204. In such instances, depending on the number of the fingers1206, the body 1204 may include a corresponding number of wings forreceiving the fingers 1206. For example, as shown, the body 1204 mayinclude a first wing 1208(1) and a second wing 1208(2) (collectivelyreferred to herein as “the wings 1208”), where the first finger 1206(1)couples to the body 1204 at the first wing 1208(1) and the second finger1206(1) couples to the body 1204 at the second wing 1208(2). In suchinstances, the first wing 1208(1) and the second wing 1208(2) may bespaced 180 degrees apart from another. By way of another example, if theend effector 1202 includes three fingers, the three fingers may beradially spaced apart from one another by 120 degrees. In suchinstances, the body 1204 of the end effector 1202 may include threewings that are radially spaced apart from one another by 120 degrees.

The fingers 1206 may pivotably couple to the wings 1208 for allowing thefingers 1206 to transition between the open position and the closedposition. For example, as shown in FIG. 12A, the fingers 1206 may becinched or positioned close together. As shown in FIG. 12B, in the openposition, the fingers 1206 may be separated or disposed apart from oneanother. The fingers 1206 may pivotably couple to the wings 1208,respectively, using bearings, bushings, and so forth. In some instances,the fingers 1206 may pivot about a rod, pin, or shaft disposed throughthe fingers 1206 and the wings 1208, respectively, and which function toadjoin the fingers 1206 to the body 1204.

The end effector 1202 (or the robotic arm 1200) may include an actuator1210 for transitioning the end effector 1202 (or the fingers 1206)between the open position and the closed position. In some instances,the actuator 1210 may include a linear actuator or a rotary actuator.For example, FIGS. 12A and 12B illustrates that a rotatory actuator(rotating actuator) may be disposed on, through, or within the body1204, and which couples to the fingers 1206 for moving the fingers 1206between the open position and the closed position.

The actuator 1210 may couple to the fingers 1206 via linkages,connectors, bars, and so forth. For example, as shown and in someinstances, the fingers 1206 may couple to the actuator 1210 via one ormore linkages, such as a first linkage 1212(1), a second linkage1212(2), and a third linkage 1212(3). In instances where the endeffector 1202 includes more than two fingers, the end effector 1202 mayinclude additional actuators and/or linkages.

The first linkage 1212(1) may couple to an end of the fingers 1206, suchas a first end 1214 of the first finger 1206(1). As shown, the firstlinkage 1212(1) may couple at a location proximally or substantiallylocated at the first end 1214 to increase a clamping force of the firstfinger 1206(1). The first linkage 1212(1) may pivotably couple to thefirst end 1214 of the first finger 1206(1) using pins and bearings, forexample. Similarly, the second linkage 1212(2) may pivotably couple tothe first end 1214 of the second finger 1206(2). The third linkage1212(3) may couple to the actuator 1210 and include opposing ends thatare coupled to the first linkage 1212(1) and the second linkage 1212(2),respectively. The connection between the third linkage 1212(3) with thefirst linkage 1212(1) and the second linkage 1212(2) may pivot to allowcomponents of the end effector 1202 to move and swivel.

As the actuator 1210 rotates, for instance, the first linkage 1212(1),the second linkage 1212(2), and the third linkage 1212(3) collectivelyoperate to transition the end effector 1202 between the open state andthe closed state. For example, when the actuator 1210 is actuated, thethird linkage 1212(3) may pull or push on the first linkage 1212(1) andthe second linkage 1212(2), respectively, for moving the fingers 1206.Whether the third linkage 1212(3) pulls or pushes on the first linkage1212(1) or the second linkage 1212(2) may depend on a direction ofrotation of the actuator 1210. For example, shown in FIG. 12A, the thirdlinkage 1212(3) may be substantially horizontal, and when rotated(clockwise or counterclockwise), the second linkage 1212(2) may besubstantially vertical. Therein, actuating the actuator 1210 in anopposite direction (counterclockwise or clockwise) may transition theend effector 1202 back to the closed position. This movement of thethird linkage 1212(3) therefore forces the first linkage 1212(1) and thesecond linkage 1212(2) to be disposed at respective positions.

The first linkage 1212(1), the second linkage 1212(2), and the thirdlinkage 1212(3) may be hingedly or pivotably coupled to one another toallow the first linkage 1212(1), the second linkage 1212(2), and thethird linkage 1212(3) to transition between the open position and theclosed position. In some instances, the first linkage 1212(1), thesecond linkage 1212(2), and/or the third linkage 1212(3) may beadjustable in length to accommodate for varying sizes of edible crowns.

The fingers 1206 may include a second end 1216, opposite the first end1214, that may engage with a stalk of the broccoli plants 104, below theedible crown of the broccoli plant when the end effector 1202transitions to the closed position. In the closed position, in someinstances, a distance 1218 may be interposed between the second ends1216 of the fingers 1206 may be such that the second ends 1216 grip, orcinch, around the stalk. In other instances, the fingers 1206 may notgrip or clasp onto the stalk such that the stalk may translate withinthe distance 1218 (X, Y, and/or Z-directions).

Additionally, in the closed position, the fingers 1206 may define aninternal space 1220 occupied by the edible crown, or which the ediblecrown is configured to reside within while being harvested and/or afterbeing harvested. The internal space 1220 may include a volume ofsufficient size to prevent the fingers 1206 bruising or otherwisedamaging the edible crown. In other words, the fingers 1206, in theclosed position, may not pinch sides of the edible crown. For example, adistance 1222 may be interposed between internal sides of the fingers1206, where the distance 1222 is of sufficient size to not pinch anddamage the edible crown. Additionally, a vertical dimension of theinternal space 1220 (Z-direction) may be of sufficient height to preventa top of the edible crown bruising against the body 1204. The internalspace 1220 of the end effector 1202 may be predetermined according to anaverage size of edible crowns that are ready for harvesting. In someinstances, a size (e.g., volume) of the internal space 1220 may beadjusted via extending the first linkage 1212(1), the second linkage1212(2), and/or the third linkage 1212(3).

The end effector 1202 is shown including a cutting mechanism 1224 forcutting the stalk of the broccoli plant to separate the edible crownfrom the rest of the broccoli plant. In some instances, the cuttingmechanism 1224 may be similar to, represent, and/or include features asdescribed above with regard to the cutting mechanism 540. The cuttingmechanism 1224 may cut the stalk when the end effector 1202 is in theclosed position, or as the end effector 1202 transitions, or istransitioning, to the closed position. In some instances, the secondends 1216 of the fingers 1206 may grip, or cinch, around the stalk tohold the stalk while the cutting mechanism 1224 cuts the stalk.

The cutting mechanism 1224 may include a blade 1236 that is configuredto rotate, spin, or swivel for cutting through a thickness of the stalk.For example, the cutting mechanism 1224 may include an actuatorconfigured to rotate the blade 1236 relative to the second finger1206(2) for cutting through the stalk. In some instances, the actuatorthat powers the blade 1236 may be air actuated or electric. The blade1236 may couple to the actuator of the cutting mechanism 1224 via asprocket, for example. When powered or instructed, the actuator maycause the blade 1236 to rotationally cut through the stalk (e.g., aboutthe Z-axis). In some instances, the blade 1236 may cut through the stalkin a direction (Y-direction) that is substantially perpendicular to adirection in which the broccoli plant grows (Z-direction). However, insome instances, the cutting mechanism 1224 may be disposed on the secondfinger 1206(2) (or other fingers) differently than shown, or may beorientated differently than shown for cutting through the stalk. Forexample, the cutting mechanism 1224 may include scissor-like orguillotine blades that converge upon one another for cutting the stalk.Additionally, or alternatively, the blade 1236 may include a saw/discthat rotates for cutting through the stalk.

As shown, the cutting mechanism 1224 may be disposed on one of thefingers 1206 (e.g., the second finger 1206(2)) for cutting the stalkbelow the edible crown, at a position proximate to the second end 1216of the fingers 1206. For example, the cutting mechanism 1224 may cut thestalk of the broccoli plant at a position just below the edible crown.As such, because the second ends 1216 of the fingers 1206 are configuredto engage the stalk at a position below the edible crown, the cuttingmechanism 1224 may cut the stalk just below the edible crown. However,the amount of stalk that remains attached to the edible crown may bevaried according to consumer preferences. Additionally, once cut, insome instances, the blade 1236 may remain in a cut position (e.g., notretracting) such that the stalk of the cut edible floret rests on theblade 1236. The blade 1236 in this position may prevent the stalk and/orthe edible crown from falling out of the fingers 1206 and/or mayotherwise secure the edible crown within the fingers 1206.

Between the first end 1214 and the second end 1216, the fingers 1206hingedly couple to the wings 1208 of the body 1204. In some instances,the wings 1208 may pivotably couple to the fingers 1206 at a positionmore proximate to the first end 1214 of the fingers 1206 than the secondend 1216 of the fingers 1206.

Meanwhile, in the open position, a distance 1226 is interposed betweenthe second ends 1216 of the fingers 1206. The distance 1226 may be ofsufficient size to allow the end effector 1202, or the fingers 1206, tofit over and around the edible crown. Once descended over the ediblecrown, the fingers 1206 of the end effector 1202 may close to secure theedible crown.

In some instances, the body 1204 may include a base end connected to therobotic arm 1200, as shown and discussed above. In some instances, therobotic arm 1200 and/or the body 1204 may couple to a positioning system(e.g., the positioning system 538) for maneuvering the end effector1202. For example, FIGS. 12A and 12B illustrate a system for positioningthe end effector 1202, which may be used in lieu of or in addition tothe robotic arm 1200 for positioning the end effector 1202.

For example, the body 1204 is shown including a base 1228 connected to acarrier 1330. The carrier 1230 may be disposed on a track or rail system1234 for translating the end effector 1202 along different axes andplanes. For example, the rail system 1234 may include slides or railsthat function to translate the end effector 1202 along the X-plane andthe Y-plane. The carrier 1230 may couple within slots of the rail system1234, or within rails of the rail system 1234. The carrier 1230 mayinclude drivers or components for translating the end effector 1202along the X-plane and the Y-plane. Such translation may allow the endeffector 1202 to be centered over the edible crown, or otherwisepositioning the end effector 1202 for harvesting the edible crown.

In some instances, the rail system 1234 may include mounts, brackets,gears, slides, tracks, motors, wheels, pulleys, pneumatics, hydrauliccylinders, cables, screw drives, turntables, or other actuators thatposition, move, or orient the end effector 1202. The components of therail system 1234 may be electric or motorized and controlled by logic orother hardware of the harvester 100 (e.g., the computing system 500),according to the position of the edible crown.

The rail system 1234, in some instances, may be coupled to a support1232 that functions to dispose the end effector 1202 at various heights.For example, the support 1232 may telescope or extend to various lengthsto position the end effector 1202 at various positions. In someinstances, the support 1232 may extend the end effector 1202 at varyingheights on/along the Z-plane. Accordingly, the combination of the railsystem 1234 and the support 1232 may position the end effector 1202 atvarious coordinate spaces for harvesting the edible crowns.Additionally, in some instances, the robotic arm 1200 may includeadditional motors that rotate or pan the end effector 1202 at varyingdegrees. This positioning may increase a grip of the end effector 1202on the edible crown.

Although the robotic arm 1200 is described as coupling or includingcertain components for positioning the end effector 1202 relative to theedible crown, other components may be included. For example, otherpneumatic systems, tracks, or slides may be used for positioning the endeffector 1202 relative to the edible crown. In such instances, the endeffector 1202 may be disposed on or coupled to a track or positioningsystem, or may couple to motors, drives, or other actuators that alignthe end effector 1202 relative to the edible crown (or a portionthereof). In some instances, however, the robotic arms 1200 may includeactuators that maneuver the end effector 1202 in horizontal and/orvertical directions (e.g., three-dimensional space) relative to theedible crown. Additionally, in some instances, the actuators may alsotilt or dispose the end effector 1202 at certain angles relative to theedible crown. Aligning the end effector 1202 in this manner may allowthe end effector 1202 to securely grip the edible crown, and withoutdamaging the grip.

In some instances, the rail system 1234 and/or the support 1232 may beconfigured to translate to account for the movement of the harvester100. For example, as the end effector 1202 grasps the edible crown inthe closed position the harvester 100 may still be moving in thedirection of travel. To prevent the end effector 1202 pulling (e.g.,tugging) on the edible crown and/or the stalk, the rail system 1234and/or the robotic arm 126 may move in an opposite direction. Thisopposite direction may be opposite to the direction of travel of theharvester 100 to keep the end effector 1202 centered over the ediblecrown while the cutting mechanism 1224 cuts the stalk. Moving the endeffector 1202 in a direction opposite to the direction of movement mayprevent pulling of the edible crown, which in turn, may prevent damageto the edible crown. In some instance, the rail system 1234 may move theend effector 1202 in the opposite direction, at the same speed as theharvester 100 is traveling.

The fingers 1206 may include a single unitary body or may be assembledfrom multiple components. In some instances, the fingers 1206 mayinclude a similar shape, contours, or features from the first end 1214to the second end 1216, or may include certain features proximate to thesecond end 1216 for receiving the edible crown. For example, below aposition at which the fingers 1206 couple to the wings 1208(Z-direction), the fingers 1206 may include features for holding theedible crown within the end effector 1202. By way of example, thefingers 1206 may include troughs, channels, flanges, or other featuresthat engage with a bottom or underneath side of the edible crown. Suchengagement may cusp, cradle, and secure the edible crown within theinternal space 1220 of the end effector 1202. In some instances, thecusping nature of the edible crown may substantially prevent the ediblecrown from rotating, rocking, or otherwise repositioning within the endeffector 1202 while being transferred to the collection point(s). Assuch, once cut, the edible crown may rest on the fingers 1206 (orportions thereof), within the internal space 1220.

The computing system 500 may communicatively couple or control therobotic arm 1200 and components thereof, such as the actuator 1210, thecutting mechanism 1224, the carrier 1230, etc. For example, thecomputing system 500 may instruct the actuator 1210 to move between theopen position and the closed position, and may instruct the cuttingmechanism 1224 to cut the stalk once the edible crown is within the endeffector 1202.

Additionally, while the above discussion relates to the linkages or thefingers 1206 being disposed at respective angles or positions, otherembodiments are envisioned. Additionally, while FIGS. 12A and 12Billustrate a certain position of the fingers 1206 in the closed positionand the open position, respectively, the fingers 1206 may be disposedfarther apart in the open position and/or spaced closer together in theclosed position, for example. The fingers 1206 may also includealternate contours as shown and described.

FIGS. 13A and 13B illustrate the flipper 542 of the harvester 100receiving edible crowns 1300 from the robotic arms 126 and transferringthe edible crowns 1300 to other portions of the harvester 100, such asconveyor belts and/or collection points.

The flipper 542 is configured to transition or move between positionsfor receiving the edible crowns 1300 and transferring the edible crowns1300. For example, FIG. 12A illustrates the flipper 542 in a firstposition or state for receiving the edible crowns 1300. In someinstances, the first position may be considered a “down position”whereby the flipper 542 receives the edible crowns 1300 from the roboticarms 126. After receiving the edible crowns 1300, the flipper 542 mayactuate and transition to a second position or state, as shown in FIG.12B. The second position may be considered an “up position” whereby theflipper 542 transfers the edible crowns 1300 to a conveyor belt forfurther processing (e.g., trimming, cleaning, etc.).

The flipper 542 includes a cradle, container, bin, or basket 1302 forreceiving the harvested edible crowns 1300. The basket 1302 may includesidewalls to secure the edible crowns 1300 within the basket 1302 andwhile transferring the edible crowns 1300. For example, as shown in FIG.13A, the harvested edible crown 1300 may reside within the basket 1302and secured therein by the sidewalls. This way the edible crown 1300 maynot roll out of the basket 1302 and onto the ground.

The basket 1302 is shown disposed at one end, or a first end, of theflipper 542. At a second end, the flipper 542 may couple to, or include,an actuator 1304. The actuator 1304 may function to transition theflipper 542 between the first position and the second position forreceiving the edible crowns 1300 and transferring the edible crowns1300. For example, as shown in FIG. 13B, the actuator 1304 may actuatethe flipper 542 upwards (Z-direction). As shown, the actuator 1304 maybe rotatable about the X-axis between the first position and the secondposition.

In some instances, the actuator 1304 may act with such speed andmovement such that at the second position, the edible crown 1300 isejected from the basket 1302. That is, from the first position, theflipper 542 may move into the second position at a sufficient speed suchthat the edible crown 1300 may be ejected from the basket 1302 and ontothe conveyor belt, for example. However, the ejection and/or speed atwhich the edible crown 1300 is ejected may be controlled and/or limitedto avoid bruising to the edible crown 1300. Furthermore, in someinstances, rather than “ejecting” the edible crown 1300, the basket 1302may be rotated over-center whereby the edible crown 1300 may fall out,roll out, or slide out of the basket 1302 and onto the conveyor belt,for example.

The transfer of the edible crown 1300 from the basket 1302 may thereforebe controlled or performed in a multitude of manners. After the basket1302 no longer contains the edible crown 1300, the actuator 1304 maytransition the flipper 542 back to the first position from receivinganother edible crown.

The basket 1302, or the flipper 542, in some instances may includesensors for determining when an edible crown is placed within the basket1302. These sensors may detect when the edible crown 1300 is within thebasket 1302 for actuating the flipper 542 and transferring the ediblecrown 1300. For example, a weight sensor or image sensor may detect thatthe edible crown 1300 is in the basket 1302, and in response, theactuator 1304 may be actuated for transition flipper 542 to the secondposition.

Additionally, or alternatively, in some instances, the actuator 1304 maybe actuated according to a schedule or predetermined interval of time.For example, the flipper 542 may receive edible crowns every threeseconds or in some other interval of time. The flipper 542 may thereforebe actuated every three seconds, regardless of whether an edible crownis within the basket 1302. The three second interval may allow enoughtime for the flipper 542 to receive an edible crown, transfer the ediblecrown, and return back to the second position for receiving anadditional edible crown. In some instances, the flipper 542 and/or thebasket 1302 may receive more than one edible crown at an instance, andaccordingly, in the second position, may transfer more than one ediblecrown.

In some instances, the flipper 542 may further include an adjustmentmechanism 1306 that adjusts a height, or length, of the basket 1302 foraccommodating different sizes of edible crowns. A guard 1308 is furtherprovided to prevent damage or debris collecting on the actuator 1304.

FIGS. 14-20 illustrate various processes related to harvesting ediblecrowns. The processes described herein are illustrated as collections ofblocks in logical flow diagrams, which represent a sequence ofoperations, some or all of which may be implemented in hardware,software, or a combination thereof In the context of software, theblocks may represent computer-executable instructions stored on one ormore computer-readable media that, when executed by one or moreprocessors, program the processors to perform the recited operations.Generally, computer-executable instructions include routines, programs,objects, components, data structures and the like that performparticular functions or implement particular data types. The order inwhich the blocks are described should not be construed as a limitation,unless specifically noted. Any number of the described blocks may becombined in any order and/or in parallel to implement the process, oralternative processes, and not all of the blocks need be executed. Fordiscussion purposes, the processes are described with reference to theenvironments, devices, architectures, diagrams, and systems described inthe examples herein, such as, for example those described with respectto FIGS. 1-13, although the processes may be implemented in a widevariety of other environments, architectures, and systems.

FIG. 14 illustrates an example process 1400 for training ML model(s),such as the ML model(s) 524. In some instances, the process 1400 may beperformed by the computing system 500 and/or the remote computingresource(s) 530, or components thereof. The ML model(s) 524 may betrained to determine trust scores related to determining whether ediblecrowns are ready for harvesting.

As discussed above, the harvester 100 may include components thatcollect and store data associated with the edible crowns. This data maybe made available to the remote computing resource(s) 530 for trainingthe ML model(s) 524. Additionally, or alternatively, the remotecomputing resource(s) 530 may store and/or maintain a databaseassociated with the edible crowns. Regardless of where the data isstored, this data may be organized within a datastore in any suitablemanner to associate individual edible crowns with relevant portions ofthe data.

At 1402, the process 1400 may train one or more machine-learningmodel(s) to identify edible crowns that are ready for harvesting. Forexample, the remote computing resource(s) 530 may include a trainingcomponent that trains the ML model(s) 524 using historical data. In someinstances, the remote computing resource(s) 530 may access a portion ofthe historical data associated with a sampled set of harvested ediblecrowns and use the sampled data to train the ML model(s) 524.Additionally, or alternatively, the historical data may includeunharvested edible crowns for training the ML model(s) 524 to identifyand label the edible crowns that are not ready for harvesting.

In some instances, the portion of the data used as training data may berepresented by a set of characteristics that is labeled with a labelindicating whether the characteristic is representative of an ediblecrown ready for harvesting. For example, if an edible crown has aparticular size, this “size” may be used as one of multiple labels for aparticular edible crown. In this manner, a supervised learning approachmay be taken to train the machine learning model(s) to predict ediblecrowns that are ready for harvesting.

In some instances, the remote computing resource(s) 530 may include atraining component configured to train ML model(s) 524 using a portionof the image data (and/or historical data) in the datastore that isassociated with a sampled set of edible crowns as training data toobtain the trained ML model(s) 524. Discussed herein, the trained MLmodel(s) 524 is/are usable by the scoring component 526 to determinescores 528 (e.g., trust scores) for a plurality of edible crowns.

Additionally, in some instances, at least some of the historical datamay have been generated from edible crowns that were previouslyharvested. For example, the historical data accessed may representedible crowns that were previously harvested and ready for harvesting,based on the edible crown exhibiting certain characteristic(s) (e.g.,size, shape, color, etc.). As part of training the ML model(s) 524, theremote computing resource(s) 530, via the training component, may labeleach characteristic of the sampled set of edible crowns with a labelthat indicates whether the characteristic is associated with an ediblecrown that is ready for harvesting or a characteristic that isassociated with an edible crown not being ready for harvesting. Examplesof labels are described herein, such as whether a size of the ediblecrown is indicative of the edible crown being ready for harvesting,whether a color of the edible crown is indicative of the edible crownbeing ready for harvesting, whether a shape of the edible crown isindicative of the edible crown being ready for harvesting, whether anamount of buds within the edible crown is indicative of the edible crownbeing ready for harvesting, and so forth. However, the labels maycorrespond to other types of characteristics.

In some instances, training the ML model(s) 524 at 1402 may includeapplying or setting weights for machine learning. These weights mayapply to a set of features derived from the historical data. In someembodiments, the weights may apply to parameters that are internal tothe ML model(s) 524 (e.g., weights for neurons in a hidden-layer of aneural network). These internal parameters of the ML model(s) 524 may ormay not map one-to-one with individual input features of the set offeatures. It is to be understood that the ML model(s) 524 may beretrained using updated historical data (e.g., the image data 522) toobtain a newly trained ML model(s) 524 that is adapted to recentlyharvested edible crowns and/or unharvested edible crowns. This allowsthe ML model(s) 524 to adapt, over time, to changing characteristics.

At 1404, the process 1400 may transmit the machine-learning model(s).For example, the remote computing resource(s) 530 may transmit, via thenetwork(s) 532, the ML model(s) 524 to the harvester 100. For example,given that the remote computing resource(s) 530 may have a computationalcapacity that exceeds that of the harvester 100, in some instances, theremote computing resource(s) 530 may train the ML model(s) 524 and thenprovide the ML model(s) 524 to the harvester 100 for use whileharvesting the edible crowns.

At 1406, the process 1400 may generate scores for the edible crown(s).For example, the scoring component 526 of the computing system 500 mayscore a plurality of edible crowns that are being imaged by the imagingsystem 516 using the ML model(s) 524. For example, the scoring component526 may access, from the computer-readable media 504 and/or receive inreal-time, the image data 522 associated with the imaged edible crowns,provide the image data 522 as input to the ML model(s) 524, andgenerate, as output from the ML model(s) 524, the scores 528 associatedwith the plurality of edible crowns. These scores 528 relate to theprobabilities of edible crowns being ready for harvesting, or not readyfor harvesting. For example, in the case of the edible crown not beingready for harvesting a “low” score may relate to the probability of theedible crown not being ready for harvesting. In the case of the ediblecrown being ready for harvesting a “high” score may relate to theprobability of the edible crown being ready for harvesting. Differentclassifiers, or identifiers, may be used or associated with those ediblecrowns that are ready for harvesting and those edible crowns that arenot ready for harvesting.

At 1408, the process 1400 may determine which edible crowns are readyfor harvesting, based at least in part on the scores. For example, withthe scores 528 determined for a plurality of edible crowns, thecomputing system 500 may be configured to determine whether the ediblecrowns are ready for harvesting. In some instances, the computing system500 may receive the scores 528 and compare these scores 528 to athreshold, or certain predetermined level. If the scores 528 are greaterthan the threshold, or satisfy the threshold, the edible crown may beflagged for harvesting. If the scores 528 are less than the threshold,or do not satisfy the threshold, the edible crown may be not be flaggedfor harvesting. In some instances, the edible crown may be flagged fornot harvesting (e.g., an indication to not harvest the flagged ediblecrown).

In some instances, rather than outputting a score, the ML model(s) 524may output indications or labels indicating whether the edible crownsare ready for harvesting and/or not ready for harvesting. For example,the indications or labels may indicate whether the edible crowns are“ready” or “not ready.” In some instances, the edible crowns not beassociated with a color, size, density, or maturity per se forcomparison with reference characteristic(s), but rather, a labelindicating whether the edible crown is ready or not ready forharvesting. As part of this process, the ML model(s) 524 may be trainedvia data associated with mature edible crowns and/or immature ediblecrowns to know how to recognize edible crowns that are ready and notready for harvesting.

FIG. 15 illustrates an example process 1500 for determining whether anedible crown is ready for harvesting, and if the edible crown is readyfor harvesting, causing the edible crown to be harvested. In someinstances, the process 1500 may be carried out or performed by theharvester 100 and/or components thereof, such as the computing system500.

Initially, in some instances, the process 1500 may begin at 1502 withde-leafing the broccoli stalk to remove leaves and isolate the ediblecrown. For example, as discussed above, removing leaves around orproximate to the edible crown may increase the imaging system 516capturing clear image(s) of the edible crown.

At 1504, the process 1500 may capture image(s) of the edible crown. Forexample, the imaging system 516 may capture one or more image(s) of theedible crown. The image(s) may be used by the harvester 100, such as thecomputing system 500, for determining whether to harvest the ediblecrown.

At 1506, the process 1500 may determine whether the edible crown isready for harvesting. For example, the computing system 500 may analyzethe image(s) captured by the imaging system 516 to determine whether theedible crown is ready for harvesting (e.g., mature). For example, the MLmodel(s) 524 may be utilized to determine whether the edible crowns areready for harvesting. Additional details of determining whether theedible crown is ready for harvesting are discussed herein with regardsto the process 1600, as shown and discussed in relation to FIG. 16. At1506, if the process 1500 determines that the edible crown is not readyfor harvesting, the process 1500 may follow the “NO” route and proceedto 1508. For example, the process 1500 may determine that the ediblecrown is not mature for harvesting (e.g., not of sufficient size, color,height, etc.).

At 1508, the process 1500 may determine to not harvest the edible crown.For example, as a result of determining that the edible crown is notready for harvesting, the harvester 100 may bypass harvesting the ediblecrown to allow the broccoli plant further time to grow and mature.Determining to not harvest the edible crown may involve the robotic arm126 refraining from actuating the end effector 536 to harvest the ediblecrown.

In some instances, the process 1500, as a result of not harvesting theedible crown, may record a location of the edible crown for use at alater instance when traveling back to the edible crown. For example, thecomputing system 500 may store information associated with theunharvested edible crowns. In instances where the edible crown is notready for harvesting, the computing system 500 may record information asto a location of the edible crown, and/or characteristic(s) of theunharvested edible crown (e.g., size, shape, color, etc.). Thesecharacteristic(s) may be used for determining when the edible crown isprojected to be ready for harvesting. For example, the characteristic(s)may indicate that the unharvested edible crown will be ready forharvesting in one week. After one week, the edible crown may then beharvested during another harvesting cycle. Furthermore, recording alocation of the unharvested edible crown may allow the harvester 100 (oran operator thereof) to precisely locate the unharvested edible crownwithin the field 102.

Alternatively, if at 1506 the process 1500 determines that the ediblecrown is ready for harvesting, the process 1500 may follow the “YES”route and proceed to 1510.

At 1510, the harvester 100 may harvest the edible crown. For example,upon determining that the edible crown is ready for harvesting, thecomputing system 500 may cause the harvester 100 (or components thereof)to actuate and harvest the edible crown.

As shown, harvesting the edible crown at 1510 may include varioussub-operations 1512-1520. For example, at 1512, harvesting the ediblecrown may include instructing the robotic arm 126 to move to a positionassociated with harvesting the edible crown (e.g., X, Y, and Zcoordinates). Once the robotic arm 126 is in position, at 1514, therobotic arm 126 may grip the edible crown via the end effector 536. Forexample, the end effector 536 may transition from the open state to theclosed state to grip the broccoli plant (e.g., the stalk) andencapsulate the edible crown. Thereafter, at 1516, the cutting mechanism540 of the robotic arm 126 may cut the stalk to separate the ediblecrown from the stalk. At 1518, the robotic arm 126 may then transfer theedible crown, which is retained within the end effector 536, to acollection site (e.g., flipper 542, conveyor belt 130, etc.). Once atthe collection site, at 1520, the robotic arm 126 may release the ediblecrown from the end effector 536 to transfer the edible crown to thecollection site.

However, although the process 1500 of harvesting the edible crown at1510 is shown and discussed as including certain operations, the roboticarm 126 (or the harvester 100) may be configured to perform any numberor series of operations for harvesting the edible crown. Furthermore,although the process 1500 is described as determining whether to harvesta single edible crown, the process 1500 may be performed for a pluralityof edible crowns across multiple rows of broccoli plants, in parallel,for harvesting edible crowns within more than one row simultaneously.

FIG. 16 illustrates an example process 1600 for determining whether toharvest an edible crown, or whether an edible crown is ready forharvesting, based on determining one or more characteristic(s) of theedible crown. The process 1600 may be carried out or performed by theharvester 100 and/or components thereof, such as the computing system500.

At 1602, the process 1600 may analyze image(s) of the edible crown. Forexample, after the imaging system 516 captures image(s) of the ediblecrown, the image(s) (color and/or depth) may be analyzed via thecomputing system 500. In some instances, the image(s) may be analyzed bythe imaging system 516 (e.g., integrated processor(s)). Analyzing theimage(s) may also include utilizing the ML model(s) 524 for determiningcertain objects or characteristics within the image(s). For example, theML model(s) 524 may analyze the image(s) to determine objects ofinterest or known objects.

At 1604, the process 1600 may determine a first characteristic of theedible crown. For example, after analyzing the image(s), the computingsystem 500 may determine a first characteristic of the edible crown. Insome instances, the first characteristic may include a size of theedible crown, a color of the edible crown, a density of the ediblecrown, a number of buds within the edible crown, and so forth. Thesecharacteristics may assist in determining whether the edible crown isready for harvesting and/or is otherwise mature for harvesting. Notedabove, the first characteristic (and other characteristics of the ediblecrown) may be determined using the ML model(s) 524 that are trained toanalyze the image(s) and identify the characteristics (e.g., asdiscussed above with regard to FIG. 14 and the process 1400) of ediblecrowns that are ready for harvesting and/or of edible crowns that arenot ready for harvesting.

At 1606, the process 1600 may compare the first characteristic with afirst reference characteristic to determine a first similarity betweenthe first characteristic and the first reference characteristic. Forexample, as part of identifying or determining the first characteristic,the process 1600 may compare the first characteristic with a referencecharacteristic for use in determining whether the edible crown is readyfor harvesting. The reference characteristic may correspond to, or beassociated with, the first characteristic for use in comparison anddetermining whether the edible crown is ready for harvesting.

For example, if the first characteristic is a color of the edible crown,the first reference characteristic may be a reference color. The colorof the edible crown may be compared against the reference color todetermine a similarity therebetween. That is, at 1606, the process 1600may determine a similarity between the color of the edible crown to areference color that is indicative of the edible crown being ready forharvesting (e.g., green). The ML model(s) 524 may determine the firstsimilarity.

In some instances, the reference color may be represented as a range ofcolors. In this sense, the reference characteristics may be associatedwith a range of values (e.g., colors, size, buds, etc.) for comparisonwith the characteristics of the edible crown. For example, the range ofcolors may span between light green and dark green, where edible crownshaving a color within this range are ready for harvesting. These“colors” within the range may be associated with certain hues, tints,shades, contrast, and/or other features for comparing the color of theedible crown. Accordingly, the color of the edible crown may not requirean “exact match” to a certain color, but rather, may fall within a rangeof colors that are indicative of the edible crown being ready forharvesting.

As such by comparing the first characteristic with the first referencecharacteristic, the process 1600 may determine a similaritytherebetween. This similarity, as alluded to above, may indicate acloseness of the first characteristic with the first referencecharacteristic, where the closeness is used to determine whether theedible crown is ready for harvesting. In some instances, as part ofdetermining the first similarity, the process 1600 may determine aconfidence of the first similarity. That is, the confidence of the firstsimilarity may represent a confidence of the first characteristic beingsimilar to, or not similar to, the first reference characteristic. Insome instances, the similarity may be represented as a score.

At 1608, the process 1600 may determine whether the first similarity isgreater than a first threshold. For example, as part of comparing thefirst characteristic to the first reference characteristic, the process1600 may determine a relatedness (i.e., how close of a match between thefirst characteristic and the first reference characteristic). If thefirst similarity is greater than the first threshold (e.g., satisfiesthe threshold), this may indicate that the first characteristic isclosely related (e.g., similar) to the first reference characteristic(and that the edible crown may be ready for harvesting). Alternatively,if the first similarity is less than first threshold (e.g., does notsatisfy the threshold), this may indicate that the first characteristicis not closely related to the first reference characteristic (and thatthe edible crown may not be ready for harvesting).

If at 1608, the process 1600 determines that the first similarity isless than the first threshold, the process 1600 may follow the “NO”route and proceed to 1610. For example, if the color of the edible crownis yellow, or a yellowish green, the process 1600 may determine thatedible crown is not ready for harvesting because the edible crown doesnot include a color representative being ready for harvesting (e.g.,light green to dark green). Noted above, as part of this determination,the process 1600 may compare hues, tints, or shades of the color of theedible crown to determine that the edible crown is not ready forharvesting. In some instances, the process 1600 may determine an averagecolor of the edible crown and/or a color within a center of the ediblecrown.

At 1610, the process 1600 may determine to not harvest the edible crown.For example, as a result of determining that the first similarity is notgreater than the first threshold, the process 1600 may determine to notharvest the edible crown. As a result of determining that the ediblecrown is not ready for harvesting, the harvester 100 may bypassharvesting the edible crown to allow the edible crown (or the broccoliplant) further time to grow and mature. Determining to not harvest theedible crown may involve the robotic arm 126 refraining from actuatingthe robotic arm 126 and the cutting mechanism 540 to harvest the ediblecrown.

Alternatively, if at 1608, the process 1600 determines that the firstsimilarity is greater than the first threshold, the process 1600 mayfollow the “YES” route and proceed to 1612. For example, if the color ofthe edible crown is green, the process 1600 may determine that theedible crown may be ready for harvesting because the edible crown has acolor that is representative of the edible crown being ready forharvesting (e.g., including a color between light green and dark green).

At 1612, the process 1600 may determine a second characteristic of theedible crown. For example, the computing system 500 may determine asecond characteristic of the edible crown, such as color, size, shape, anumber of buds, etc. The second characteristic may be another of thecharacteristics of the edible crown, different than the firstcharacteristic at 1604. By way of example, the second characteristic maybe a size of the broccoli (e.g., area, largest cross-sectionaldimension, diameter, height, etc.).

At 1614, the process 1600 may compare the second characteristic with asecond reference characteristic to determine a second similarity betweenthe second characteristic and the second reference characteristic. Forexample, as part of identifying or determining the secondcharacteristic, the process 1600 may compare the second characteristicwith a reference characteristic for use in determining whether theedible crown is ready for harvesting. The reference characteristic maycorrespond to, or be associated with, the second characteristic fordetermining whether the edible crown is ready for harvesting. Forexample, the second reference characteristic may be a reference size(e.g., reference area, reference cross-sectional dimension, referencediameter, reference height, etc.).

The size of the edible crown may be compared against the reference sizeto determine a similarity therebetween. That is, at 1614, the process1600 may determine a similarity between the size of the edible crown toa reference size that is indicative of the edible crown being ready forharvesting. In some instances, the similarity may be represented as ascore.

In some instances, the reference size may be represented as a range ofsizes. For example, the range of sizes may span between 4.0 inches and5.75 inches, where edible crowns having a size within this range may beready for harvesting. In some instances, however, the reference size maybe associated with a single value, such that if edible crowns aregreater than 4.0 inches, the edible crowns may be deemed ready forharvesting. As such by comparing the second characteristic with thesecond reference characteristic, the process 1600 may determine asimilarity therebetween, where this similarity may indicate a closenessof the second characteristic with the second reference characteristic.

In some instances, as part of determining the second similarity, theprocess 1600 may determine a confidence of the second similarity. Thatis, the confidence of the second similarity may represent a confidenceof the second characteristic being similar to, or not similar to, thesecond reference characteristic.

At 1616, the process 1600 may determine whether the second similarity isgreater than a second threshold. For example, as part of comparing thesecond characteristic to the second reference characteristic, theprocess 1600 may determine a relatedness (i.e., how close of a matchbetween the second characteristic and the second referencecharacteristic). If the second similarity is greater than the secondthreshold (e.g., satisfies the threshold), this may indicate that thesecond characteristic is closely related (e.g., similar) to the secondreference characteristic. Alternatively, if the second similarity isless than second threshold (e.g., does not satisfy the threshold), thismay indicate that the second characteristic is not closely related tothe second reference characteristic.

If at 1616, the process 1600 determines that the second similarity isless than the second threshold, the process 1600 may follow the “NO”route and proceed to 1610. For example, if the size of the edible crownis less than 4.0 inches, the process 1600 may determine that ediblecrown is not ready for harvesting because the edible crown is not ofsufficient size for harvesting.

Alternatively, if at 1616, the process 1600 determines that the secondsimilarity is greater than the second threshold, the process 1600 mayfollow the “YES” route and proceed to 1618. For example, if the size ofthe edible crown is 5.25, the process may determine that the ediblecrown may be ready for harvesting because the edible crown has a sizethat is representative of an edible crown being ready for harvesting.

At 1618, the process 1600 may harvest the edible crown. For example, thecomputing system 500 may instruct components of the harvester 100 (e.g.,robotic arm 126) to harvest the edible crown. In some instances, bycomparing two characteristics of the edible crown against referencecharacteristics, before harvesting the edible crown, the process 1600may be confidence that the edible crown is mature or ready forharvesting. That is, by comparing multiple characteristics of the ediblecrowns with characteristics representative of edible crowns ready forharvesting, the process 1600 may be sure, or confident, (e.g., thethresholds) that the edible crowns are ready for harvesting.

However, as shown, in some instances, the process 1600 may proceed to1618 from 1608 after determining that the first characteristic isgreater than the first threshold similarity. For example, in someinstances, only one characteristic may be used to determine that theedible crown is ready for harvesting. For example, if the process 1600has a certain confidence that the edible crown is ready for harvesting,after analyzing the first characteristic, the process 1600 may determineto harvest the edible crown. For example, in some instances, the process1600 may proceed to 1618 from 1608 if the process 1600 is confident ofthe first similarity above a threshold confident level. The process 1600may therefore be confident of the color of the edible crown, and thatthe edible crown is therefore ready for harvesting. As a result, theprocess 1600 may cause the edible crown to be harvested.

In some instances, the process 1600, from 1608 may proceed to 1612 ifthe process 1600 has a confidence that is less than the certainconfidence level. For example, if at 1608 the process 1600 determinesthat the color of the edible crown is green, but is not that confidentin the decision (e.g., below a threshold confidence level), the process1600 may proceed to 1612 for analyzing the second characteristic of theedible crown to determine whether to harvest the edible crown.

In some instances, the process 1600 may dynamically determine the firstcharacteristic and/or the second characteristic for the edible crowns.For example, in analyzing the image(s), the computing system 500 mayidentify which characteristics of the edible crowns are mostidentifiable, or which characteristics the computing system 500 has thehighest confidence. In this sense, in some instances, the computingsystem 500 may identify a first characteristic of a first edible crownand a second characteristic of the first edible crown, which may bedifferent than a first characteristic of a second edible crown and asecond characteristic of the second edible crown. For example, thecomputing system 500 may analyze color and size for the first ediblecrown to determine whether the first edible crown is ready forharvesting, and may analyze color and the number of buds for the secondedible crown to determine whether the second edible crown is ready forharvesting. In some instances, however, the first characteristic and thesecond characteristic may be same across the edible crowns. For example,the first characteristic may be color and the second characteristic maybe size. Here, the process 1600 may determine the color and the size ofthe edible crown to determine whether the edible crown is ready forharvesting.

Furthermore, while the FIG. 16 and the process 1600 are illustrated anddescribed as comparing two characteristics for determining whether theedible crown is ready for harvesting, more than or less than twocharacteristics may be used. For example, noted above, the process 1600may only use a single characteristic, such as size or color, todetermine whether the edible crown is ready for harvesting by comparingthe size or color to reference values. The size or the color of theedible crown, alone, in some instances, may be dispositive (or above acertain confidence threshold) that the edible crown is ready forharvesting. Using a single characteristic, as opposed to multiple, mayincrease a harvesting rate of the harvester.

Alternatively, the process 1600 may compare three, four, five, etc.characteristics for determining whether the edible crown is ready forharvesting. In some instances, the process 1600 may analyze additionalcharacteristics of the edible crown in instances where the process 1600has a confidence less than a certain threshold. For example, if theprocess 1600 is confident less than a certain threshold that the colorof the edible crown is representative of an edible crown ready forharvesting, the process 1600 may analyze an additional characteristic ofthe edible crown for determining whether the edible crown is ready forharvesting.

Although the process 1600 is discussed herein as determiningcharacteristics of the edible crown, in some instances, the process 1600may determine characteristics of other portions of the broccoli plant ofruse in determining whether the edible crown (or the broccoli plant) isready for harvesting. For example, the process 1600 may analyze theimage(s) to determine a thickness of the stalk, a height of the broccoliplant, and/or a size of the leaves for use in determining whether theedible crown is mature and ready for harvesting. Although the process1600 is described as determining whether to harvest a single ediblecrown, the process 1600 may be performed for a plurality of ediblecrowns across multiple rows of broccoli, in parallel, for harvestingedible crowns within more than one row simultaneously.

FIG. 17 illustrates an example process 1700 for aligning the endeffector with an edible crown and harvesting the broccoli edible crown.

At 1702, the process 1700 may determine that an edible crown is readyfor harvesting. For example, based at least in part on analyzing imagedata associated with the edible crown, the computing system 500 maydetermine that the edible crown is ready for harvesting.

At 1704, the process 1700 may determine coordinates of the edible crownfor harvesting the edible crown. For example, after determining toharvest the edible crown, the computing system 500 may determinecoordinates that are associated with harvesting the edible crown. Todetermine the coordinates, in some instances, the computing system 500may analyze the image data 522, the location data 548, and/or theencoder data 546. In some instances, the coordinates may represent acenter point of the edible crown (e.g., X, Y, and Z positions), and/ormay be relative to components of the harvester 100 that function toharvest the edible crown (e.g., the end effector 536).

At 1706, the process 1700 may align the end effector along a first axisassociated with a first coordinate of the coordinates. For example, therobotic arm (e.g., the robotic arm 126 or the robotic arm 1200) and/orthe positioning system (e.g., the positioning system 538 and/or the railsystem 1234) may move to align the end effector along an X-axis/planethat is associated with an X-coordinate position of the edible crown.

At 1708, the process 1700 may align the end effector along a second axisassociated with a second coordinate of the coordinates. For example, therobotic arm and/or the positioning system may move to align the endeffector along a Y-axis/plane that is associated with a Y-coordinateposition of the edible crown. In some instances, the steps 1706 and 1708may be performed substantially simultaneously and/or in reverse order.After 1706 and 1708, the end effector 536 may be substantially centeredabove (e.g., disposed vertically above) the edible crown. In someinstances, a center of the end effector may be centered with a center ofthe edible crown in multiple directions, or along multiple planes (e.g.,X and Y planes).

At 1710, the process 1700 may descend the end effector along a thirdaxis associated with a third coordinate of the coordinates. For example,the robotic arm and/or a support (e.g., the support 1232) may move toalign the end effector along a Z-axis/plane that is associated with aZ-coordinate position of the edible crown. For example, after beingcentered above the edible crown, and aligned with the X and Y coordinatepositions of the edible crown, the end effector 536 may descend upon theedible crown to align within a Z-plane extending through the centerpoint of the edible crown.

At 1712 the process 1700 may close fingers of the end effector aroundthe edible crown. For example, once the end effector 536 is centered onthe edible crown, the fingers (e.g., the fingers 1206) may be actuatedvia an actuator (e.g., the actuator 1210) for enclosing the fingersaround the edible crown. After closing the fingers, the edible crown maybe cradled and/or supported within an interior (e.g., the interior space1220) of the end effector 536. In some instances, the computing system500 may cause the fingers 1206 to close and/or instruct the actuator toactuate and enclose the edible crown.

At 1714, the process 1700 may cut the stalk to separate the ediblecrown. For example, after the edible crown is enclosed within the endeffector, a cutting mechanism (e.g., the cutting mechanism 540 and/orthe cutting mechanism 1224) may actuate to cut the stalk and separatethe edible crown from a remaining portion of the stalk. After being cut,the edible crown may remain encased in the end effector, within thefingers.

At 1716, the process 1700 may transfer the edible crown to a flipper.For example, after being harvested, the robotic arm and/or thepositioning system may move to transfer the edible crown to the flipper(e.g., the flipper 542). At the flipper 542, the end effector 536 maytransition to the open position to transfer the edible crown to theflipper 542. Therein, the edible crown may be transferred to otherlocations on the harvester 100 to be cleaned and/or further processed.

FIG. 18 illustrates an example process 1800 for adjusting harvestingcoordinates of an edible crown that is ready for harvesting. Initially,as noted above, at 1802, the process 1800 may determine or receive anindication that an edible crown is ready for harvesting (e.g., based onprocessing the image data 522).

At 1804, the process 1800 may determine first coordinates associatedwith harvesting the edible crown. For example, in some instances, thefirst coordinates may be determined based at least in part on analyzingthe encoder data 546 as received from the encoder 544 and/or the imagedata 522 as generated by the imaging system(s) 516. In some instances,the encoder data 546 may be used to determine a distance traveled by theharvester 100 within the field, and correspondingly, may be used todetermine a location (e.g., the first coordinates) of the edible crown.Additionally, or alternatively, the image data 522 (e.g., generated bythe IR sensor(s) 520) may be used to determine a location of the ediblecrown ready for harvesting.

At 1806, the process 1800 may receive GPS coordinates of a harvesterthat harvests the edible crowns. For example, the navigational system508 of the harvester 100 may include a GPS component for determining GPScoordinates of the harvester 100. Such GPS coordinates may indicate aposition of the harvester 100 within the field, while the firstcoordinates represent a position of the edible crown within the field.However, knowing the location of the harvester 100 within the field (viathe GPS coordinates) allows for a position of the edible crown to bedetermined. For example, knowing the relative position of the imagingsystem 516 on the harvester 100 allows for the conversion of the GPScoordinates of the harvester 100 to GPS coordinates of the edible crown.

In some instances, the encoder data 546 used for determining a distancetraveled by the harvester 100, which is used to determine the firstcoordinates, may be subject to inaccuracies. For example, debris mayaccumulate on the wheels 108 of the harvester 100 and the wheels 108 mayoccasionally slip (e.g., lose traction). Furthermore, the accumulationof debris may impact a radius of the wheel 108, which may in turn impacta determined distance traveled by the harvester 100 (using the encoderdata 546). As such, discussed herein, the first coordinates associatedwith harvesting the edible crown may be adjusted to account forinaccuracies.

For example, at 1810, the process 1800 may compare the GPS coordinateswith the first coordinates. In some instances, comparing the GPScoordinates with the first coordinates may be used to determine an errorin the encoder data 546, and the error can be used to determine anadjustment to be applied when determining the coordinates of the ediblecrown. In this sense, determining whether there is an error may involvecomparing the GPS coordinates and the first coordinates in order tocompensate for the error in the encoder output.

At 1810, the process 1800 may determine whether to adjust the firstcoordinates. For example, the computing system 500 may compare the firstcoordinates with the GPS coordinates to determine whether the firstcoordinates are different than or similar to the GPS coordinates. Insome instances, the difference and/or similarity may be represented at athreshold, or predetermined amount. If the difference, for example, isgreater than a threshold then the first coordinates may not be adjusted.If, however, the difference is less than a threshold (e.g., the firstcoordinates and the GPS coordinates are substantially similar), then thefirst coordinates may not be adjusted. For example, if the difference isless than a threshold, the process 1800 may determine to not adjust thefirst coordinates and the process 1800 may follow the “NO” route andproceed to 1812.

At 1812, the process 1800 may harvest the edible crown using the firstcoordinates. For example, the computing system may transmit instructionsto the robotic arm, the positioning system, and/or other components toharvest the edible crown based on the first coordinates.

Alternatively, if at 1810 the process 1800 determines to adjust thefirst coordinates (e.g., the difference is greater than a threshold),the process 1800 may follow the “YES” route and proceed to 1814.

At 1814, the process 1800 may determine second coordinates associatedwith harvesting the edible crowns. For example, in some instances, thesecond coordinates may be based on the GPS coordinates to account for anadjustment in the encoder data (e.g., as mud builds up on the wheels 108of the harvester). Here, the second coordinates may represent anadjustment applied to the first coordinates when determining the finalposition coordinates of the edible crown to compensate for the error inthe encoder output.

As another example, the image data 522 from the imaging system 516 maybe processed using an object detection algorithm to track objects (e.g.,rocks, edible crowns, etc.). This object detection may be used todetermine a distance traveled by the harvester 100 over a series ofimages. This image-based distance determination may additionally, oralternatively, be used to determine an error in the encoder data 546,and to determine an adjustment (or offset) to be applied whendetermining the position coordinates of the edible crowns.

At 1816, the process 1800 may harvest the edible crown using the secondcoordinates.

Although the process 1800 illustrates determining the coordinates forharvesting the edible crown using the encoder data, and determiningwhether to adjust the coordinates using the image data and/or the GPScoordinates, in some instances, the coordinates for harvesting theedible crown may be determining using one or more of the image data, theencoder data, an/or the GPS coordinates. For example, the position forharvesting the edible crown may be determined solely from the GPScoordinates and/or solely from the image data. In these instances, theprocess 1800 may track a position of the harvester 100 and/or distancetraveled by the harvester 100 in a forward direction of travel, whichmay then be used for determining coordinates of unharvested ediblecrowns (and which are ready for harvesting). Therein, the coordinatesare used to control the end effectors.

FIG. 19 illustrates an example process 1900 for adjusting a speed of theharvester 100 based on a number or ratio of harvestable edible crownsacross multiple rows. In some instances, the harvester 100 may adjust inspeed based on the number of harvestable and/or unharvestable ediblecrowns across all rows of broccoli plants that are being harvested bythe harvester 100. In this sense, as discussed herein, the harvester 100may travel as fast as the “slowest” row, or the row having the greatestnumber of harvestable edible crowns.

Initially, the harvester 100, via the imaging system 516 may imageedible crowns of broccoli plants being harvested across the multiplerows. For example, at 1902, the process 1900 may capture first image(s)of first edible crowns within a first row of broccoli plants. At 1904,the process 1900 may capture second image(s) of second edible crownswithin a second row of broccoli plants. At 1906, the process 1900 maycapture n^(th) image(s) of n^(th) edible crowns within a n^(th) row ofbroccoli plants. In some instances, the operations 1902, 1904, and 1906may be performed at the same time as the harvester 100 moves about afield while the harvester 100 is harvesting edible crowns.

After capturing the image(s), the process 1900 may determine a number ofharvestable edible crowns within each row of the broccoli plants. Forexample, at 1908, the process 1900 may determine a first number ofedible crowns within the first row of broccoli plants that are ready forharvesting. As discussed in detail hereinabove, the process 1900 maydetermine the first number of edible crowns to be harvested based onanalyzing the first image(s) of the individual broccoli plants withinthe first row and determining the number of edible crowns within thefirst row that are ready for harvesting. For example, the process 1900,at 1908, may determine that three edible crowns within the first row areready for harvesting.

At 1910, the process 1900 may determine a second number of edible crownswithin the second row of broccoli plants that are ready for harvesting.The process 1900 may determine the second number of edible crowns to beharvested based on analyzing the second image(s) of the individualbroccoli plants and therein, determining the number of edible crownswithin the second row that are ready for harvesting. For example, theprocess 1900, at 1910, may determine that two edible crowns within thesecond row are ready for harvesting.

At 1912, the process 1900 may determine an n^(th) number of ediblecrowns within the n^(th) row of broccoli plants that are ready forharvesting. The process 1900 may determine the n^(th) number of ediblecrowns to be harvested based on analyzing the n^(th) image(s) of theindividual broccoli plants and then determining the number of ediblecrowns within the n^(th) row that are ready for harvesting. For example,the process 1900, at 1912, may determine that zero edible crowns withinthe second row are ready for harvesting.

After the number of harvestable edible crowns are determined within eachrow, at 1914, the process 1900 may determine the greatest number ofedible crowns within the first row, the second row, and the n^(th) row.For example, the process 1900 may compare the harvestable edible crownswithin the first row, the second row, and the n^(th) row (e.g., thefirst number, the second number, the n^(th) number). Determining whichrow of the broccoli plants contains the greatest number of harvestableedible crowns may therefore be used for adjusting the speed of theharvester to account for harvester 100 harvesting all of the harvestableedible crowns. For example, continuing with the above example, as thefirst row has three harvestable edible crowns, the second row has twoharvestable edible crowns, and the n^(th) row has zero harvestableedible crowns, the process 1900 may determine that the greatest numberof edible crowns ready for harvesting is three.

From 1914, the process 1900 may proceed to 1916 whereby the harvester100 may adjust in speed to accommodate for the greatest number ofharvestable edible crowns. For example, the harvester 100 may slow downin speed to allow the robotic arm(s) 126 of the first row to harvest allof the harvestable edible crowns within the first row. At this same timethough, the edible crowns within the second row and the n^(th) row maybe harvested by respective robotic arm(s) 126. However, the fastest theharvester 100 may travel is a speed that allows enough time for therobotic arm(s) 126 of the first row to harvest every edible crown withinthe first row. This means that the harvester 100 may travel at a speedthat accommodates the robotic arms 126 in the first row to pick everyharvestable edible crown, even if the other robotic arms 126 for theother rows remain relatively idle at times, as there may be relativelyfewer harvestable edible crowns in the other rows.

Therefore, in some instances, if there is at least one row of broccoliplants with mostly harvestable edible crowns, the speed of the harvester100 may be decreased to travel slower over to ensure that the roboticarms 126 for the row with mostly harvestable edible crowns is affordedenough time to harvest all of the edible crowns. Traveling at a higherspeed, for example, may not afford enough time for the robotic arms 126to harvest all of the edible crowns that are ready for harvesting, andas a result, the harvester 100 may pass over certain edible crowns thatare ready for harvesting.

By way of another example, if there are relatively few edible crownsthat are ready for harvesting across all rows, the speed of theharvester 100 may be increased to travel faster.

In some instances, the harvester 100 may determine the number ofharvestable edible crowns across all rows that are being harvested, maydetermine ratios of harvestable to unharvestable edible crowns, and/or,may determine a number of consecutive harvestable/unharvestable ediblecrowns in a given row for use in adjusting the travel speed of theharvester 100.

It should be understood that the process 1900 may continuously, anddynamically, determine the number of harvestable edible crowns withineach row as the harvester 100 moves about the field. That is, as theharvester 100 moves, broccoli plants within each row will come into afield of view of the imaging systems 516 associated with each row ofbroccoli. Therein, the imaging systems 516 image the edible crowns, anddetermine, in real-time, the number of edible crowns to be harvested (orwhich are ready for harvesting). As suggested, the harvester 100 maydetermine the number of harvestable edible crowns, per row, and acrossany number of n^(th) rows (e.g., six, ten, eight, twelve, etc.). Theprocess 1900 continuously determines whether to adjust the speed of theharvester 100, in real-time, based on the number of harvestable ediblecrowns across the rows of broccoli plants.

As such, because broccoli plants mature at different rates, on a givenday, some edible crowns in the field may be ready to harvest, whileother edible crowns in the field may not be ready to harvest. Theprocess 1900 may therefore provide efficiency gains by enabling theharvester 100 to travel faster at times when the robotic arms 126 wouldotherwise remain relatively idle (when there are a few number of ediblecrowns to be harvested), and slower at times when there are a greaternumber of edible crowns to be harvested.

FIG. 20 illustrates an example process 2000 for determining a type ofcut for harvesting edible crowns. In some instances, depending oncharacteristic(s) of the edible crown to be harvested, the harvester 100may harvest the edible crown in different ways. In this sense, theharvester 100 may determine a type of cut, or operations for performingtypes of cuts, depending on the characteristic(s) of the edible crown tobe harvested.

At 2002, the process 2000 may determine a size of an edible crown to beharvested. For example, after determining that an edible crown is readyfor harvesting, the computing system 500 may determine a size of theedible crown. The size of the edible crown may correspond to a largestcross-sectional dimension, an average cross-sectional dimension, adiameter, an area, volume, and so forth.

In some instances, determining the size of the edible crown may involveanalyzing image(s) captured by the imaging system 516 for determiningdimensions of the edible crown, such as width (X-direction), length(Y-direction), and/or height (Z-direction). In some instances, inanalyzing the image(s) the computing system 500 may determine suchdimensions of the edible crown. For example, the computing system 500may determine that the edible crown is five inches wide. In someinstances, the width (or other dimensions) may be an average width or agreatest cross-sectional dimension. For example, edible crowns oftengrow irregular and are not symmetrical. The edible crown may thereforebe ovular, egg-shaped, hexagonal, circular, and/or any combinationthereof. In determining the size of the edible crown, the computingsystem 500 may determine a greatest cross-sectional dimension among aplurality of cross-sectional dimension (e.g., random sampling ofcross-sectional dimensions). The computing system 500 then determine thegreatest cross-sectional dimension for determining the size of theedible crown. In some instances, the computing system 500 may determinean average cross-sectional dimension for use in determining the size ofthe edible crown.

At 2004, the process 2000 may determine whether the size of the ediblecrown is within a first range of sizes. For example, the computingsystem 500 may compare the size of the edible crown with the first rangeof sizes, or bounds thereof, for determining whether the size of theedible crown is within the first range of sizes. By way of example, thefirst range of sizes may be from 4.0 and 4.75 inches. Accordingly, ifthe size of the edible crown is 5 inches, the computing system 500 maydetermine that the size of the edible crown is not within the firstrange of sizes.

In instances where the size of the edible crown is within the firstrange of sizes, the process 2000 may follow the “YES” route and proceedto 2006.

At 2006, in response to determining that the size of the edible crown iswithin the first range of sizes, the process 2000 may select a firsttype of cut for harvesting the edible crown. For example, as notedabove, depending on the size of the edible crown, the harvester 100 mayperform different types of cuts, or different operations for harvestingthe edible crown. In some instances, the first type of cut may includegrasping the edible crown and cutting the stalk with the cuttingmechanism 540. Therein, the edible crown may be harvested andtransferred to other components of the harvester 100.

Alternatively, if at 2004, the process 2000 determines that the size ofthe edible crown is not within the first range of sizes, the process2000 may follow the “NO” route and proceed to 2008.

At 2008, the process 2000 may determine that the size of the ediblecrown is within a second range of sizes. For example, as a result of notbeing within the first range of sizes, the process 2000 may determinethat the size is within a second range of sizes, such as between 4.75and 5.75 inches.

At 2010, the process 2000 may select a second type of cut for harvestingthe edible crown. For example, the computing system 500 may determine toharvest the edible crown using a second type of cut that is differentthan the first type of cut. In some instances, the second type of cutmay include stripping the leaves around the edible crown and thencutting the edible crown. For example, once the end effector 536 graspsaround the edible crown (with the fingers 1206), the robotic arm 126 (orother actuators) may advance the end effector downward towards theground to strip or peel back leaves that may be adjacent to the ediblecrown. Stripping back the leaves in this manner may reduce processingtime at later operations to clean or remove leaves. After movingdownward, the robotic arm 126 may pull the end effector back up to aposition where the cutting mechanism 540 severs the stalk below theedible crown.

As such, the harvester 100 may perform different operations forharvesting the edible crown based on a size of the edible crown.Although the process 2000 is illustrated and described herein asincluding two types of cuts, the harvester 100 may be configured forperforming more types of cuts, such as three or four. In some instances,these types of cuts may be dependent on the size of the edible crown(e.g., cross-sectional dimension), the shape of the edible crown, aheight of the edible crown, or other characteristics. As the size of theedible crown may equate to leaves or other foliage being around theedible crown, these types of cuts may serve to reduce post-processing ofthe edible crown (e.g., removing leaves, cleaning, etc.). In someinstances, the type of cut performed may be dependent upon a length ofstalk desired to be attached (or remaining attached) to the ediblecrown.

FIGS. 21A-21E illustrate detailed views of a robotic arm 2100 of theharvester 100. In some instances, the robotic arm 2100 may be similar toand/or include features as described above with regard to the roboticarm 126 and/or the robotic arm 2100. Additionally, the robotic arm 2100may be usable with the processes described above. The robotic arm 2100includes an end effector 2102. In some instances, the end effector 2102may be similar to and/or include features as described above with regardto the end effector 536 and/or the end effector 2102. The end effector2102 is configured to transition between an open position.

As similarly discussed above with regard to the end effector 536 and/orthe end effector 2102, the end effector 2102 may represent a gripperthat grasps edible crowns of broccoli plants ready for harvesting. Inthe open position, the end effector 2102 may descend unto or over ediblecrowns. In the closed position, the end effector 2102 may grasp ontoedible crowns (or portions of the stalk) for retaining edible crownswithin the end effector 2102.

As shown, and similar to the end effector 2102, the end effector 2102may include a body 2104 and fingers 2106 attached to the body 2104. Insome instances, the end effector 2102 may include two fingers 2106.However, in some instances, the end effector 2102 may include more thantwo fingers, such as three fingers, four fingers, and/or any othernumber of fingers. The fingers 2106 may pivotably couple to the body2104 for allowing the fingers 2106 to transition between the openposition and the closed position. Additionally, the end effector 2102(or the robotic arm 2100) may include an actuator for transitioning theend effector 2102 (or the fingers 2106) between the open position andthe closed position. In some instances, the actuator may include alinear actuator or a rotary actuator. The actuator may couple to thefingers 2106 via linkages, connectors, bars, and so forth. For example,as shown and in some instances, the fingers 2106 may couple to theactuator via one or more linkages.

The fingers 2106 may engage with a stalk of the broccoli plants, belowthe edible crown of the broccoli plant when the end effector 2102transitions to the closed position. In the closed position, the fingers2106 may define an internal space occupied by the edible crown, or whichthe edible crown is configured to reside within while being harvestedand/or after being harvested.

The end effector 2102 is shown including a cutting mechanism 2108 forcutting the stalk of the broccoli plant to separate the edible crownfrom the rest of the broccoli plant. In some instances, the cuttingmechanism 2108 may be similar to and/or include features as describedabove with regard to the cutting mechanism 540. However, as shown, thecutting mechanism 2108 may be different than the cutting mechanism 1224as discussed above with regard to the end effector 2102.

In some instances, the cutting mechanism 2108 may include a first blade2110 disposed on an end of a first of the fingers 2106 and a secondblade 2112 disposed on a second of the fingers 2106. In some instances,the first blade 2110 and the second blade 2112 may represent a doubleguillotine blade-like cutter that, when the end effector 2102 enclosesthe edible crown, the second blade 2112 actuates to cut the stalk. Insome instances, the cutting mechanism 2108 may include an air actuatoror an electric motor that extends the second blade 2112 towards thefirst blade 2110 to cut through the stalk. For example, referring toFIG. 21E, when the end effector 2102 is in the closed state, an opening2114 may be defined at least partially between the first blade 2112 andthe second blade 2112. As the end effector 2102 closes, the stalk of thebroccoli plant may be positioned within the opening, while the ediblecrown may be disposed within an interior of the fingers 2106. Therein,the actuator of the cutting mechanism 2108 may actuate the second blade2112 in the Y-direction, towards the first blade 2110. This actuationmay sever the edible crown from the stalk. As such, the first blade 2110and the second blade 2112 may function as a guillotine-style cutter forsevering the edible crown. In some instances, the actuator may linearlyactuate the second blade 2112 towards the first blade 2110. As shown inFIGS. 21D and 21E, for example, the first blade 2110 and/or the secondblade 2112 may be semi-circular in shape to be disposed around aperimeter of the stalk when the end effector 2102 closes.

In some instances, after cutting the stalk, the second blade 2112 mayremain in a cut position such that the second blade 2112 is disposedwithin the opening 2114, with the stalk resting on a surface of thesecond blade 2112. The stalk may rest on the second blade 2112 toprevent the edible crown repositioning within the end effector 2102.

In some instances, the body 2104 may include a base end connected to therobotic arm 2100. In some instances, the robotic arm 2100 and/or thebody 2104 may couple to a positioning system 2116 (e.g., the positioningsystem 538) for maneuvering the end effector 2102. In some instances,the positioning system 2116 may include mounts, brackets, gears, slides,tracks, motors, wheels, pulleys, pneumatics, hydraulic cylinders,cables, screw drives, turntables, or other actuators that position,move, or orient the end effector 2102. The positioning system 2116, insome instances, may also extend to various lengths to position the endeffector 2102 at various positions. Although the robotic arm 2100 isdescribed as coupling or including certain components for positioningthe end effector 2102 relative to the edible crown, other components maybe included.

The fingers 2106 may include a single unitary body or may be assembledfrom multiple components. Additionally, the fingers 2106 may includetroughs, channels, flanges, or other features that engage with a bottomor underneath side of the edible crown. Such engagement may cusp,cradle, and secure the edible crown within an internal space of the endeffector 2102. Moreover, the computing system 500 may communicativelycouple or control the robotic arm 2100 and components thereof, such asthe actuator, the cutting mechanism 2108, etc.

While various examples and embodiments are described individuallyherein, the examples and embodiments may be combined, rearranged, andmodified to arrive at other variations within the scope of thisdisclosure.

Although the subject matter has been described in language specific tostructural features and/or methodological acts, it is to be understoodthat the subject matter defined in the appended claims is notnecessarily limited to the specific features or acts described. Rather,the specific features and acts are disclosed as illustrative forms ofimplementing the claims.

1. A machine for harvesting edible crowns of broccoli plants, themachine comprising: an imaging device; a global positioning system(GPS); an end effector; one or more processors; and one or morecomputer-readable media storing instructions that, when executed, causethe one or more processors to perform operations comprising: determininga route of the machine within a field for harvesting the edible crownsof the broccoli plants, the route including a starting position and anending position; receiving, from the imaging device, image datarepresenting an edible crown along the route that is ready forharvesting; receiving, from the GPS, location data representing aposition of the machine within the field; determining, based at least inpart on the image data, first position coordinates that are associatedwith the edible crown; determining, based at least in part on thelocation data and the first position coordinates, second positioncoordinates that are associated with harvesting the edible crown; andinstructing the end effector to move to the second position coordinatesfor harvesting the edible crown.
 2. The machine of claim 1, theoperations further comprising: receiving, from the imaging device,second image data representing a second edible crown along the routethat is ready for harvesting; receiving, from the GPS, second locationdata representing a second position of the machine within the field, thesecond position being different than the position; determining, based atleast in part on the second image data, third position coordinates thatare associated with the second edible crown; determining, based at leastin part on the second location data and the third position coordinates,fourth position coordinates that are associated with harvesting thesecond edible crown; and instructing the end effector to move to thefourth position coordinates for harvesting the second edible crown. 3.The machine of claim 1, the operations further comprising: determining,based at least in part on the image data, a first location of an objectof interest at a first time; receiving, from the imaging device, secondimage data representing a second location of the object of interest at asecond time that is after the first time; and determining a distancetraveled by the machine between the first time and the second time basedat least in part on the first location and the second location, andwherein determining the second position coordinates are further based atleast in part on the distance traveled by the machine.
 4. The machine ofclaim 1, further comprising at least one wheel and an encoder configuredto track a rotational movement of the at least one wheel, the operationsfurther comprising receiving, from the encoder, encoder datarepresenting a distance traveled by the machine from the startingposition to a position associated with the edible crown, and whereindetermining the second position coordinates are further based at leastin part on the distance traveled by the machine.
 5. The machine of claim1, further comprising at least one wheel and an encoder configured totrack a rotational movement of the at least one wheel, wherein theposition of the machine comprises a first position, the operationsfurther comprising: receiving, from the encoder, encoder datarepresenting a distance traveled by the machine from the startingposition to a position associated with the edible crown; determining,based at least in part on the encoder data, a second position of themachine within the field; comparing the first position with the secondposition; determining a difference between the first position and thesecond position; and adjusting the second position based at least inpart on the first position.
 6. A machine comprising: an imaging device;a driving mechanism; one or more processors; and one or morecomputer-readable media storing instructions that, when executed, causethe one or more processors to perform operations comprising: determininga route of the machine within a field for harvesting edible crowns;instructing the driving mechanism to travel along the route; receiving,from the imaging device, image data representing an edible crown; anddetermining, based at least in part on the image data, that the ediblecrown is ready for harvesting and one or more coordinate positionsassociated with harvesting the edible crown.
 7. The machine of claim 6,further comprising an end effector for harvesting the edible crown, theoperations further comprising instructing the end effector to move tothe one or more coordinate positions, wherein at the one or morecoordinate positions, the end effector is configured to grasp the ediblecrown and cut the edible crown from a stalk.
 8. The machine of claim 6,wherein the image data comprises first image data, the locationcomprises a first location, the operations further comprising:receiving, from the imaging device, second image data representing anobject of interest at a first time; receiving, from the imaging device,third image data representing the object of interest at a second timethat is after the first time; and determining, based at least in part onthe object of interest, a distance traveled by the machine between thefirst time and the second time, and wherein determining the one or morecoordinate positions is based at least in part on the distance traveledby the machine.
 9. The machine of claim 6, further comprising an endeffector, wherein the one or more coordinate positions comprise a firstcoordinate position, a second coordinate position, and a thirdcoordinate position, the operations further comprising: instructing, ata first time, the end effector to move to the first coordinate position;instructing, at a second time that is after the first time, the endeffector to move to the second coordinate position, wherein at the firstcoordinate position and the second coordinate position, the end effectoris disposed vertically above the edible crown; and instructing, at athird time that is after the second time, the end effector to move tothe third coordinate position to descend upon the edible crown.
 10. Themachine of claim 6, further comprising at least one wheel and an encoderconfigured to track a rotational movement of the at least one wheel, theoperations further comprising: receiving, from the encoder, encoder datarepresenting a distance traveled by the machine along the route; anddetermining, based at least in part on the distance traveled by themachine, a location of the machine within the field, and whereindetermining the one or more coordinate positions is based at least inpart on the location of the machine within the field.
 11. The machine ofclaim 6, further comprising a navigational system communicativelycoupled to the driving mechanism, the operations further comprisingreceiving, from the navigational system, location data corresponding toa location of the machine within the field, and wherein determining theone or more coordinate positions associated with harvesting the ediblecrown is based at least in part on the location data.
 12. The machine ofclaim 11, the operations further comprising: receiving, from the imagingdevice or an additional imaging device, additional image datarepresenting an additional edible crown ready for harvesting; receiving,from the navigational system, additional coordinates corresponding to anadditional location of the machine within the field; and determining,based at least in part on at least one of the additional image data orthe additional coordinates, one or more additional coordinate positionsassociated with harvesting the additional edible crown.
 13. A methodcomprising: receiving first data from an encoder representing a distancetraveled by a machine; receiving second data from an imaging devicerepresenting an edible crown; determining, based at least in part on thesecond data, that the edible crown is ready for harvesting; anddetermining, based at least in part on the first data and the seconddata, position coordinates of the edible crown.
 14. The method of claim13, further comprising instructing a harvesting component of the machineto move to the position coordinates to harvest the edible crown.
 15. Themethod of claim 14, wherein instructing the harvesting component to moveto the position coordinates comprises: instructing, at a first time, theharvesting component to grip the edible crown; instructing, at a secondtime that is after the first time, the harvesting component to cut theedible crown from a stalk; and instructing, at a third time that isafter the second time, the harvesting component to transfer the ediblecrown to a collection location.
 16. The method of claim 13, wherein: thefirst data is used to determine a rotational movement of at least onewheel of the machine for determining the distance traveled by themachine; and the distance traveled by the machine is in reference to astarting position of a route along which the machine is configured totravel, the edible crown being located along the route.
 17. The methodof claim 13, wherein the distance traveled by the machine comprises afirst distance traveled, the method further comprising: analyzing thesecond data to determine a first position of an object of interest;receiving third data from the imaging device that includes the object ofinterest at a second position; comparing a first position and the secondposition to determine a second distance traveled by the machine; anddetermining a difference between the first distance traveled and thesecond distance traveled, and wherein determining the positioncoordinates of the edible crown is further based at least in part on thedifference.
 18. The method of claim 17, wherein the differencerepresents an adjustment to the first distance traveled.
 19. The methodof claim 13, further comprising receiving third data representing GlobalPositioning System (GPS) coordinates associated with a position of themachine, and wherein determining the position coordinates of the ediblecrown is based at least in part on the GPS coordinates.
 20. The methodof claim 13, wherein the position coordinates comprise first positioncoordinates, the method further comprising: receiving third data from aGlobal Positioning System (GPS) component representing GPS coordinatesof the machine; determining a location of the machine within the fieldbased at least in part on the GPS coordinates; determining an adjustmentto the first position coordinates based at least in part on the GPScoordinates; and determining, based at least in part on the adjustment,third position coordinates of the edible crown.